Pith. sign in

REVIEW 4 major objections 4 minor 103 references

Listen, See and Track: Spatio-Temporal Audio-Visual Sound Event Reasoning for Omni-Modal Language Models

T0 review · 4 major / 4 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper argues that dynamic sound scenes require tracking each source's identity, position, and motion over time, and presents a model and benchmark that do so.

desk verdict ST-OmniQA is a well-designed benchmark for a real gap, but the headline numbers rest on an unvalidated LLM judge that also serves as the RL reward model, so treat the quantitative claims as conditional until the evaluator and artifacts are sorted out. read the letter →

arxiv 2608.09435 v1 pith:DNZFGKYL submitted 2026-08-10 cs.AI

Zhi Zeng , Cheng Zhang , Zesheng Yang , Rendong Pi , Jiaying Wu , Di Zhang , Zihan Ma , Guodong Li
show 4 more authors
Zhou Yang Yu Xiang Yifei Zheng Minnan Luo
This is my paper · ORCID
classification cs.AI
keywords spatio-temporalaudio-visualreasoningfirst-orderAmbisonicssoundeventlocalizationanddetectionmultimodallanguagemodelstrajectorytrackingquestionansweringcurriculumlearningreinforcement
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Understanding a dynamic sound scene means deciding what emits each sound, where the source is, how it moves, and which visible object it belongs to; the paper argues this is a tracking-and-binding problem, not a clip-level classification problem. To evaluate this capability, the paper builds ST-OmniQA, a synthetic benchmark of 40,000 panoramic videos with synchronized first-order Ambisonics audio and 400,000 question-answer pairs organized into four levels, from single-source sound-event recognition to multi-source trajectory reasoning and visual-instance binding. The paper then proposes ST-Omni-R1, which represents each clip as one semantic audio token plus 40 time-ordered trajectory tokens and fuses those with panoramic visual tokens inside a language-model decoder, trained by progressive curriculum learning and reasoning-tree reinforcement learning. The reported result is 77.83% average semantic accuracy on ST-OmniQA against 37.28% for the best evaluated baseline, with transfer gains on three public spatial-audio benchmarks.

What carries the argument

The load-bearing object is the STA-encoder's split audio representation: a single semantic token for clip-level event content and $K=40$ trajectory tokens that retain time-varying direction, distance, and activity. The input is a seven-channel FOA-derived feature tensor combining channel-wise log-Mel spectrograms with Mel-projected acoustic-intensity components, processed by an Audio Spectrogram Transformer; temporal self-attention over frequency-pooled patch features produces the ordered trajectory tokens. Their initialization loss $L_{\text{traj}}$ supervises per-bin binary activity, unit direction vectors, and log-distance, while a frozen static encoder preserves event semantics. These audio tokens are projected by a trainable connector and concatenated with video and question tokens as $C = [V_{\text{tok}}; A_{\text{tok}}; Q_{\text{tok}}]$, which lets the language model bind acoustic trajectories to visible instances. The benchmark uses the constraint $|C_A|>1$, $|C_V|>1$, $|C_{AV}|=1$ so that a Level-D answer is unique only after cross-modal binding.

What would settle it

Take a few hundred held-out ST-OmniQA answers from all four levels and have human annotators score semantic equivalence instead of the automated judge; if the human scores change the ranking or shrink the gap, the headline claim fails. As a second check, ablate the 40 trajectory tokens from the model's audio input; if Level C and Level D accuracy do not fall, the trajectory mechanism is not carrying the result.

Watch

Extended reading notes

Core claim

The central claim is that time-varying source geometry is the missing ingredient in audio-language and vision-language models, and that it can be learned and expressed through FOA-derived semantic and trajectory representations. In the benchmark, each source carries a multimodal state $S_i(t) = \{e_i, a_i(t), g_i(t), m_i(t), v_i(t), o_i, R_i(t)\}$, where $g_i(t)$ holds azimuth, elevation, and source-listener distance and $R_i(t)$ holds relations to landmarks, occluders, and other sources. ST-Omni-R1's STA-encoder turns a four-channel FOA waveform into a semantic token $s$ and $K=40$ temporally ordered trajectory tokens $\tau_1,\dots,\tau_K$, supervised during initialization by per-bin activity, unit direction, and log-distance targets; the connector inserts these into the decoder context together with video tokens. Progressive curriculum stages and reasoning-tree reinforcement learning train the model, which reaches 84.70%, 76.20%, 55.70%, and 94.70% semantic accuracy on the four levels and a 77.83% average, and outperforms the BAT baseline on the TAU-NIGENS, L3DAS22, and STARSS23 transfer sets. Semantic accuracy here means whether a generated answer conveys the same meaning as the reference.

Load-bearing premise

Every reported accuracy number is produced by a fixed automated semantic-equivalence judge with no reported human agreement, error analysis, or per-category validation; if that judge is lenient or biased toward ST-Omni-R1's answer style, the headline gap and the transfer numbers are not established.

Editorial extensions

If this is right

  • General-purpose audio-visual and audio-only models that compress a clip into global event semantics will miss source-level geometry; ST-OmniQA puts this gap at 37.28% best baseline versus 77.83%.
  • The learned spatial and motion token sequence transfers to real-world audio and audio-visual spatial datasets, so the representation is not tied only to the synthetic benchmark.
  • Because Level-D questions are constructed so that neither modality alone determines the answer, strong performance on that level is evidence of genuine cross-modal binding rather than unimodal shortcutting.
  • Progressive curriculum stages and reasoning-tree RL both add capability, with Stage II lifting the hardest scene-grounded level from 90.90% to 94.70%.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the benchmark scenes are rendered from indoor 3D meshes with simulated room acoustics, real-world deployment could face a domain gap that the three transfer datasets only partially cover.
  • The single automated semantic-equivalence judge makes the reported gap contingent on that judge's scoring behavior; a human-validated scoring subset would be the most direct check, and the paper does not report one.
  • The trajectory-token interface could be reused outside question answering, for example as a generic audio-state module for embodied agents that must localize and follow a moving sound source; the paper does not make that claim.
  • Because each question is generated from executable reasoning graphs over structured scene states, the benchmark could be extended to harder compositional or physical-reasoning questions beyond the four levels covered.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper introduces ST-OmniQA, a synthetic benchmark built from 40K panoramic videos with synchronized first-order Ambisonics (FOA) audio, containing 400K question-answer pairs organized into four capability levels that cover source perception, multi-source localization, trajectory relations, and audio-visual binding. The paper also proposes ST-Omni-R1, a model that combines an STA-encoder producing one semantic token and 40 trajectory tokens with panoramic visual context, and trains it with progressive curriculum learning followed by reasoning-tree reinforcement learning. The authors report 77.83% average semantic accuracy on ST-OmniQA versus 37.28% for the best evaluated baseline, and additional transfer results on TAU-NIGENS, L3DAS22, and STARSS23.

Significance. If the results hold, the benchmark and model fill a genuine gap: treating sound sources as persistent, spatially located, moving entities that must be bound to visible instances, rather than as clip-level acoustic events. The paper's strengths include the modality-necessity constraint in Eq. (2), which reduces unimodal shortcuts; the scene-room-level split that prevents leakage; the executable reasoning graphs behind Level C/D traces; and the transfer evaluation on three public spatial-audio benchmarks. The central weakness is that all reported accuracy numbers are produced by a single unvalidated LLM judge that also participates in the reward computation during training, so the quantitative claims are not yet established. The manuscript is worth pursuing after the evaluation pipeline is independently validated and statistical variability is reported.

major comments (4)
  1. [Experiments, "Evaluation protocol" paragraph] All semantic-accuracy numbers in Tables 2-4 are computed by DeepSeek-v4-flash, yet no human agreement, per-category validation, or error analysis is reported. Because every headline comparison passes through this judge, the 77.83% versus 37.28% gap and the transfer numbers are properties of the judge as much as of the models. Please report human agreement on a representative sample, per-level and per-question-type accuracy, and a judge-free exact-match check for structured fields such as event class, DoA, distance, and motion state.
  2. [Stage II, Eq. (14) and the following sentence] The same semantic-equivalence evaluator that produces the reported accuracy is used to score free-form responses inside the RT-RL reward in Eq. (14). The model is therefore optimized directly toward the judge's notion of semantic equivalence, and any stylistic preference of the judge is amplified during training; the Level D jump from 90.90 (SFT) to 94.70 (SFT+RT-RL) in Table 4 may reflect judge alignment rather than reasoning skill. Please validate the judge independently, evaluate with a different judge or human raters, and report the agreement between RT-RL reward scores and human judgments.
  3. [Experiments, "Implementation details" and Tables 2-4] No seed variance, error bars, or significance tests are reported, despite small differences such as the Level C comparison of 55.70 versus 54.40 in Table 4. Please report at least three seeds with standard deviations, or bootstrap confidence intervals, for the main comparisons so that the reader can assess whether the reported gaps are statistically meaningful.
  4. [Experiments, "Main Results" and Tables 2-3] The baseline comparisons are not controlled: general-purpose baselines are evaluated without task-specific tuning and do not receive spatial audio, while BAT retains its own free-form prompting protocol, and on the transfer benchmarks BAT scores are near zero on several axes. The claim of superiority over the best baseline is therefore not established under a common protocol. Please add at least one spatial-audio-capable baseline under the same input and evaluation protocol, or present an explicit audio-only controlled comparison that reports both models with matched modalities.
minor comments (4)
  1. [References] References for Hyun-Bin et al. 2026 and Oh et al. 2026 cite the same arXiv identifier (2606.14141) but are listed as separate works; please merge or disambiguate them.
  2. [Experiments, footnote 1] Footnote 1 states that additional details appear in the Appendix, but no appendix is included in the manuscript; please either add the appendix or remove the pointer.
  3. [Experiments, general] The paper does not state whether the benchmark, evaluation prompts, or model weights will be released; given the benchmark's intended reuse, an availability statement would be valuable.
  4. [References] The reference for Yang, Liu, and Li 2022 contains the typo "Acouts." in the venue name; please correct it to "Acoustics".

Circularity Check

1 steps flagged · score 6.0 of 10

Headline semantic accuracy is judged by the same DeepSeek-v4-flash evaluator used as the free-form reward in RT-RL, closing the evaluation loop.

  1. fitted input called prediction [Section 'Stage II: Reasoning-Tree Reinforcement Learning' (Eq. 14); 'Experiments / Evaluation protocol']
    "Fields with annotation-defined targets are evaluated directly against the corresponding ST-OmniQA annotations, whereas free-form responses are scored using a fixed semantic-equivalence evaluator to accommodate valid linguistic variation. ... Semantic accuracy is evaluated by DeepSeek-v4-flash (Xu et al. 2026)."

    The same unvalidated DeepSeek-v4-flash semantic-equivalence evaluator appears twice: in Eq. 14 it scores free-form responses during GRPO reward computation, and in the Evaluation protocol it defines the reported semantic accuracy for all models. ST-Omni-R1's free-form answers are therefore token-level optimized (Eq. 15) to maximize exactly the scores that are later reported as its accuracy. Baselines were not optimized against this judge, so the 77.83% vs. 37.28% gap and transfer numbers partly measure how well ST-Omni-R1 has learned the judge's equivalence and stylistic preferences rather than independent semantic skill. No human agreement or judge-free validation is reported to break the loop, so the 'prediction' is a re-measurement of the training reward.

full rationale

ST-OmniQA is constructed from simulator states with deterministic queries and scene-disjoint training/test splits; training on those annotations is ordinary supervised learning, not circular. The STA-encoder derivation (Eqs. 4-10) defines features and losses from FOA physics and benchmark geometry without reusing the target metric. The transfer experiments use external real-world datasets, although their scores also pass through the same DeepSeek judge. The one load-bearing circularity-adjacent step is the identity between the free-form reward evaluator in Stage II and the semantic-accuracy evaluator in the experiments. Because the model is optimized against this judge and then judged by it, with no human agreement or independent validation, the central quantitative claim is not fully self-contained. This is an evaluation-loop confound rather than a derivation that reduces to its inputs, so the score is moderate rather than extreme.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

ST-OmniQA is a benchmark artifact, not a new physical entity, and trajectory tokens are model internals, so the invented-entities ledger is empty. The free parameters are all design choices or unreported weights. The axioms are the load-bearing assumptions about simulation fidelity, the LLM judge, and template structure.

free parameters (4)
  • K trajectory tokens = 40
    Number of temporally ordered audio tokens; chosen by hand in Eq. 6 and used throughout.
  • STA-encoder loss weights λ_sem, λ_evt = not reported
    Balance trajectory, semantic distillation, and event classification in Eq. 10; values are not given.
  • RT-RL reward weights λ_fmt, λ_tree, λ_ans, λ_con = not reported
    Only the sum constraint λ_fmt+λ_tree+λ_ans=1 is stated in Eq. 14; individual values are not reported.
  • Curriculum training schedule = one epoch per stage at LR 1e-5, batch 128
    The four-stage curriculum and its duration are hand-chosen; Stage II uses 8 samples per prompt at LR 1e-6.
assumptions (4)
  • domain assumption SoundSpaces2.0 and Matterport3D simulations faithfully represent real-world audio-visual source binding
    The entire benchmark is synthetic; if acoustics or panoramas diverge from real scenes, high benchmark accuracy may not imply real-world capability.
  • domain assumption DeepSeek-v4-flash semantic accuracy is a valid proxy for human correctness
    The evaluation protocol uses this judge for all accuracy numbers without reporting human agreement or error analysis.
  • domain assumption QA templates and executable reasoning graphs produce unique answers without answer shortcuts
    The modality-necessity filter for Level D reduces unimodal shortcuts, but template regularity and RL rewards built from the same annotations may leak structure.
  • standard math Standard optimization and audio-visual background (AST, GRPO, next-token cross-entropy) is valid
    Training objectives rely on established methods; no new mathematical claims are made.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Listen, See and Track: Spatio-Temporal Audio-Visual Sound Event Reasoning for Omni-Modal Language Models." pith.science (2026). https://pith.science/paper/DNZFGKYL

@misc{pith2026260809435,
  author       = {Pith},
  title        = {Pith review of: Listen, See and Track: Spatio-Temporal Audio-Visual Sound Event Reasoning for Omni-Modal Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DNZFGKYL}},
  note         = {Machine review of arXiv:2608.09435}
}
read the original abstract

Understanding dynamic sound sources requires jointly determining what produces a sound, where the source is located, and how it moves over time. Yet existing audio-language models often represent clips as global acoustic events, while vision-language models lack the spatial audio cues needed to localize and track individual sources. To evaluate this missing capability, we introduce ST-OmniQA, a spatio-temporal audio-visual question-answering benchmark built from panoramic videos paired with synchronized first-order Ambisonics (FOA) audio of moving sound sources. It contains 40K videos and 400K question-answer pairs organized into four capability levels covering sound-event recognition, direction of arrival, source distance, motion trajectories, and temporally grounded audio-visual reasoning. Building on this benchmark, we propose ST-Omni-R1, which integrates FOA-derived semantic and trajectory representations with panoramic visual context and is trained through progressive curriculum learning and reasoning-tree reinforcement learning. ST-Omni-R1 achieves 77.83\% average semantic accuracy across the four levels, compared with 37.28\% for the best evaluated baseline. Results on three public spatial-audio benchmarks further indicate that its learned spatial and motion representations transfer beyond ST-OmniQA.

Figures

Figures reproduced from arXiv: 2608.09435 by the authors.

Figure 1
Figure 1. Representative ST-OmniQA tasks spanning four capability groups: (a) single-source audio-visual grounding; (b) [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Overview of ST-Omni-R1. • Level C: Spatio-temporal relational reasoning. Time￾conditioned spatial relations, temporal ordering, and tra￾jectory comparisons across sources. • Level D: Scene-grounded audio-visual reasoning. Source-instance binding, landmark grounding, occlusion understanding, visibility transitions, and tracking after vi￾sual disappearance. These levels specify complementary evaluation capabilities ra… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

103 extracted references · 53 canonical work pages

  1. [1]

    Proceedings of the 6th Workshop on Detection and Classification of Acoustic Scenes and Events (DCASE 2021) , pages =

    A Dataset of Dynamic Reverberant Sound Scenes with Directional Interferers for Sound Event Localization and Detection , author =. Proceedings of the 6th Workshop on Detection and Classification of Acoustic Scenes and Events (DCASE 2021) , pages =. 2021 , doi =

    2021

  2. [2]

    arXiv preprint arXiv:2606.19348 , year=

    Deepseek-v4: Towards highly efficient million-token context intelligence , author=. arXiv preprint arXiv:2606.19348 , year=

    arXiv

  3. [3]

    arXiv preprint arXiv:2504.18425 , year=

    Kimi-audio technical report , author=. arXiv preprint arXiv:2504.18425 , year=

    arXiv

  4. [4]

    doi:10.21437/Interspeech.2021-698 , issn =

    Yuan Gong and Yu-An Chung and James Glass , year =. doi:10.21437/Interspeech.2021-698 , issn =

    doi:10.21437/interspeech.2021-698 2021

  5. [5]

    Advances in neural information processing systems , volume=

    STARSS23: An audio-visual dataset of spatial recordings of real scenes with spatiotemporal annotations of sound events , author=. Advances in neural information processing systems , volume=

  6. [6]

    arXiv preprint arXiv:2402.03300 , year=

    Deepseekmath: Pushing the limits of mathematical reasoning in open language models , author=. arXiv preprint arXiv:2402.03300 , year=

    arXiv

  7. [7]

    ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , pages=

    L3DAS22 challenge: Learning 3D audio sources in a real office environment , author=. ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , pages=. 2022 , organization=

    2022

  8. [8]

    Communication, Simulation, and Intelligent Agents: Implications of Personal Intelligent Machines for Medical Education

    Clancey, William J. Communication, Simulation, and Intelligent Agents: Implications of Personal Intelligent Machines for Medical Education. Proceedings of the Eighth International Joint Conference on Artificial Intelligence (IJCAI-83)

Show all 103 references
  1. [9]

    Classification Problem Solving

    Clancey, William J. Classification Problem Solving. Proceedings of the Fourth National Conference on Artificial Intelligence

  2. [10]

    , title =

    Robinson, Arthur L. , title =. 1980 , doi =. https://science.sciencemag.org/content/208/4447/1019.full.pdf , journal =

    1980

  3. [11]

    New Ways to Make Microcircuits Smaller---Duplicate Entry

    Robinson, Arthur L. New Ways to Make Microcircuits Smaller---Duplicate Entry. Science

  4. [12]

    Clancey and Glenn Rennels , abstract =

    Diane Warner Hasling and William J. Clancey and Glenn Rennels , abstract =. Strategic explanations for a diagnostic consultation system , journal =. 1984 , issn =. doi:https://doi.org/10.1016/S0020-7373(84)80003-6 , url =

    1984 doi

  5. [13]

    arXiv preprint arXiv:2406.07476 , year=

    Videollama 2: Advancing spatial-temporal modeling and audio understanding in video-llms , author=. arXiv preprint arXiv:2406.07476 , year=

    arXiv

  6. [14]

    5-omni technical report , author=

    Qwen2. 5-omni technical report , author=. arXiv preprint arXiv:2503.20215 , year=

    arXiv

  7. [15]

    arXiv preprint arXiv:2407.10759 , year=

    Qwen2-audio technical report , author=. arXiv preprint arXiv:2407.10759 , year=

    arXiv

  8. [16]

    arXiv preprint arXiv:2511.21631 , year=

    Qwen3-VL Technical Report , author=. arXiv preprint arXiv:2511.21631 , year=

    arXiv

  9. [17]

    5-vl technical report , author=

    Qwen2. 5-vl technical report , author=. arXiv preprint arXiv:2502.13923 , year=

    arXiv

  10. [18]

    arXiv preprint arXiv:2306.02858 , year=

    Video-llama: An instruction-tuned audio-visual language model for video understanding , author=. arXiv preprint arXiv:2306.02858 , year=

    arXiv

  11. [19]

    arXiv preprint arXiv:2303.08774 , year=

    Gpt-4 technical report , author=. arXiv preprint arXiv:2303.08774 , year=

    arXiv

  12. [20]

    and Rennels, Glenn R

    Hasling, Diane Warner and Clancey, William J. and Rennels, Glenn R. and Test, Thomas. Strategic Explanations in Consultation---Duplicate. The International Journal of Man-Machine Studies

  13. [21]

    Poligon: A System for Parallel Problem Solving

    Rice, James. Poligon: A System for Parallel Problem Solving

  14. [22]

    Transfer of Rule-Based Expertise through a Tutorial Dialogue

    Clancey, William J. Transfer of Rule-Based Expertise through a Tutorial Dialogue

  15. [23]

    The Engineering of Qualitative Models

    Clancey, William J. The Engineering of Qualitative Models

  16. [24]

    2023 , eprint=

    Attention Is All You Need , author=. 2023 , eprint=

    2023

  17. [25]

    Pluto: The 'Other' Red Planet

    NASA. Pluto: The 'Other' Red Planet

  18. [26]

    2023 , volume =

    Han, Zongbo and Zhang, Changqing and Fu, Huazhu and Zhou, Joey Tianyi , journal = IEEE_J_PAMI, title =. 2023 , volume =

    2023

  19. [27]

    2022 , volume =

    Yang, Bing and Liu, Hong and Li, Xiaofei , booktitle = ICASSP, title =. 2022 , volume =

    2022

  20. [28]

    2023 , pages =

    Yabo Wang and Bing Yang and Xiaofei Li , title =. 2023 , pages =

    2023

  21. [29]

    PILOT: Introducing Transformers for Probabilistic Sound Event Localization , author =

  22. [30]

    Evidential Deep Learning to Quantify Classification Uncertainty , year =

    Sensoy, Murat and Kaplan, Lance and Kandemir, Melih , booktitle = NIPS, pages =. Evidential Deep Learning to Quantify Classification Uncertainty , year =

  23. [31]

    2023 , publisher =

    Uncertainty-inspired open set learning for retinal anomaly identification , author =. 2023 , publisher =

    2023

  24. [32]

    Tbrats: Trusted brain tumor segmentation , author =

  25. [33]

    2023 , volume =

    Wang, Qingwang and Yin, Cheng and Song, Haochen and Shen, Tao and Gu, Yanfeng , journal = IEEE_J_GRSL, title =. 2023 , volume =

    2023

  26. [34]

    2021 , pages =

    Bao, Wentao and Yu, Qi and Kong, Yu , booktitle = ICCV, title =. 2021 , pages =

    2021

  27. [35]

    2015 , volume =

    Panayotov, Vassil and Chen, Guoguo and Povey, Daniel and Khudanpur, Sanjeev , booktitle = ICASSP, title =. 2015 , volume =

    2015

  28. [36]

    NOISEX-92: A database and an experiment to study the effect of additive noise on speech recognition systems , journal =

    Assessment for automatic speech recognition: II. NOISEX-92: A database and an experiment to study the effect of additive noise on speech recognition systems , journal =. 1993 , doi =

    1993

  29. [37]

    2019 , volume =

    Adavanne, Sharath and Politis, Archontis and Nikunen, Joonas and Virtanen, Tuomas , journal = IEEE_J_STSP, title =. 2019 , volume =

    2019

  30. [38]

    2022 , volume =

    An, Inkyu and Kwon, Youngsun and Yoon, Sung-eui , journal = IEEE_J_RO, title =. 2022 , volume =

    2022

  31. [39]

    2023 , volume =

    Richter, Julius and Welker, Simon and Lemercier, Jean-Marie and Lay, Bunlong and Gerkmann, Timo , journal = IEEE_J_ASLP, title =. 2023 , volume =

    2023

  32. [40]

    Industrial fluid pipeline leak detection and localization based on a multiscale Mann-Whitney test and acoustic emission event tracking , journal = MSSP, volume =

    Zahoor Ahmad and Tuan-Khai Nguyen and Akhand Rai and Jong-Myon Kim , keywords =. Industrial fluid pipeline leak detection and localization based on a multiscale Mann-Whitney test and acoustic emission event tracking , journal = MSSP, volume =. 2023 , doi =

    2023

  33. [41]

    and Carter, G

    Knapp, C. and Carter, G. , journal = IEEE_J_ASSP, title =. 1976 , volume =

    1976

  34. [42]

    , journal = IEEE_J_AP, title =

    Schmidt, R. , journal = IEEE_J_AP, title =. 1986 , volume =

    1986

  35. [43]

    2010 , volume =

    Raspaud, Martin and Viste, Harald and Evangelista, Gianpaolo , journal = IEEE_J_ASLP, title =. 2010 , volume =

    2010

  36. [44]

    1968 , publisher =

    A generalization of Bayesian inference , author =. 1968 , publisher =

    1968

  37. [45]

    2018 , publisher =

    Subjective Logic: A formalism for reasoning under uncertainty , author =. 2018 , publisher =

    2018

  38. [46]

    Attention is All you Need , year =

    Vaswani, Ashish and Shazeer, Noam and Parmar, Niki and Uszkoreit, Jakob and Jones, Llion and Gomez, Aidan N and Kaiser, ukasz and Polosukhin, Illia , booktitle = NIPS, pages =. Attention is All you Need , year =

  39. [47]

    , journal = IEEE_J_ASLP, title =

    Diaz-Guerra, David and Miguel, Antonio and Beltran, Jose R. , journal = IEEE_J_ASLP, title =. 2021 , volume =

    2021

  40. [48]

    Habets, Emanuel A. P. and Cohen, Israel and Gannot, Sharon , title =. 2008 , volume =. doi:10.1121/1.2987429 , issn =

    2008 doi

  41. [49]

    , journal = IEEE_J_ASLP, title =

    Diaz-Guerra, David and Miguel, Antonio and Beltran, Jose R. , journal = IEEE_J_ASLP, title =. 2023 , volume =

    2023

  42. [50]

    and Evers, Christine and Schmidt, Alexander and Mellmann, Heinrich and Barfuss, Hendrik and Naylor, Patrick A

    Löllmann, Heinrich W. and Evers, Christine and Schmidt, Alexander and Mellmann, Heinrich and Barfuss, Hendrik and Naylor, Patrick A. and Kellermann, Walter , booktitle = SAM, title =. 2018 , volume =

    2018

  43. [51]

    and Moore, AlastairH

    Löllmann, Heinrich W. and Moore, AlastairH. and Naylor, Patrick A. and Rafaely, Boaz and Horaud, Radu and Mazel, Alexandre and Kellermann, Walter , booktitle =. Microphone array signal processing for robot audition , year =

  44. [52]

    2024 , volume =

    Zhu, Xin-Cheng and Zhang, Hong and Feng, Hui-Tao and Zhao, Deng-Huang and Zhang, Xiao-Jun and Tao, Zhi , journal = IEEE_J_IM, title =. 2024 , volume =

    2024

  45. [53]

    2019 , volume =

    Pak, Junhyeong and Shin, Jong Won , journal = IEEE_J_ASLP, title =. 2019 , volume =

    2019

  46. [54]

    2020 , volume =

    Varzandeh, Reza and Adiloğlu, Kamil and Doclo, Simon and Hohmann, Volker , booktitle = ICASSP, title =. 2020 , volume =

    2020

  47. [55]

    2020 , volume =

    Comanducci, Luca and Borra, Federico and Bestagini, Paolo and Antonacci, Fabio and Tubaro, Stefano and Sarti, Augusto , journal = IEEE_J_ASLP, title =. 2020 , volume =

    2020

  48. [56]

    , journal = IEEE_J_ASLP, title =

    Nguyen, Thi Ngoc Tho and Gan, Woon-Seng and Ranjan, Rishabh and Jones, Douglas L. , journal = IEEE_J_ASLP, title =. 2020 , volume =

    2020

  49. [57]

    2020 , volume =

    Comanducci, Luca and Cobos, Maximo and Antonacci, Fabio and Sarti, Augusto , booktitle = ICASSP, title =. 2020 , volume =

    2020

  50. [58]

    arXiv preprint arXiv:2501.12948 , year=

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning , author=. arXiv preprint arXiv:2501.12948 , year=

    arXiv

  51. [59]

    arXiv preprint arXiv:2507.06261 , year=

    Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities , author=. arXiv preprint arXiv:2507.06261 , year=

    arXiv

  52. [60]

    2025 , eprint=

    GLM-4.1V-Thinking: Towards Versatile Multimodal Reasoning with Scalable Reinforcement Learning , author=. 2025 , eprint=

    2025

  53. [61]

    2021 , volume =

    He, Weipeng and Motlicek, Petr and Odobez, Jean-Marc , journal = IEEE_J_ASLP, title =. 2021 , volume =

    2021

  54. [62]

    and Nguyen, Ngoc Khanh and Jones, Douglas L

    Nguyen, Thi Ngoc Tho and Watcharasupat, Karn N. and Nguyen, Ngoc Khanh and Jones, Douglas L. and Gan, Woon-Seng , journal = IEEE_J_ASLP, title =. 2022 , volume =

    2022

  55. [63]

    2021 , volume =

    Cheng, Longbiao and Sun, Xingwei and Yao, Dingding and Li, Junfeng and Yan, Yonghong , journal = IEEE_J_ASLP, title =. 2021 , volume =

    2021

  56. [64]

    2022 , volume =

    SongGong, Kunkun and Wang, Wenwu and Chen, Huawei , journal = IEEE_J_ASLP, title =. 2022 , volume =

    2022

  57. [65]

    2022 , volume =

    Lee, Soo Young and Chang, Jiho and Lee, Seungchul , journal = IEEE_J_IM, title =. 2022 , volume =

    2022

  58. [66]

    2021 , volume =

    Krause, Daniel and Politis, Archontis and Kowalczyk, Konrad , booktitle = EUSIPCO, title =. 2021 , volume =

    2021

  59. [67]

    2022 , volume =

    Zhong, Tianle and Velázquez, Israel Mendoza and Ren, Yi and Meana, Héctor Manuel Pérez and Haneda, Yoichi , booktitle = ICASSP, title =. 2022 , volume =

    2022

  60. [68]

    , title =

    Neal, Radford M. , title =. 1996 , isbn =

    1996

  61. [69]

    Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning , author =

  62. [70]

    Nature reviews neuroscience , volume=

    Multisensory integration: current issues from the perspective of the single neuron , author=. Nature reviews neuroscience , volume=. 2008 , publisher=

    2008

  63. [71]

    Current Biology , volume=

    Neural processing of auditory looming in the human brain , author=. Current Biology , volume=. 2002 , publisher=

    2002

  64. [72]

    Current biology , volume=

    The ventriloquist effect results from near-optimal bimodal integration , author=. Current biology , volume=. 2004 , publisher=

    2004

  65. [73]

    Current Biology , volume=

    When correlation implies causation in multisensory integration , author=. Current Biology , volume=. 2012 , publisher=

    2012

  66. [74]

    Proceedings of the IEEE conference on computer vision and pattern recognition , pages=

    Learning to localize sound source in visual scenes , author=. Proceedings of the IEEE conference on computer vision and pattern recognition , pages=

  67. [75]

    Proceedings of the European conference on computer vision (ECCV) , pages=

    Audio-visual scene analysis with self-supervised multisensory features , author=. Proceedings of the European conference on computer vision (ECCV) , pages=

  68. [76]

    European conference on computer vision , pages=

    Soundspaces: Audio-visual navigation in 3d environments , author=. European conference on computer vision , pages=. 2020 , organization=

    2020

  69. [77]

    IEEE Transactions on Audio, Speech and Language Processing , year=

    PSELDNets: Pre-trained neural networks on a large-scale synthetic dataset for sound event localization and detection , author=. IEEE Transactions on Audio, Speech and Language Processing , year=

  70. [78]

    arXiv preprint arXiv:2606.14141 , year=

    Spatio-Temporal Audio Language Modeling for Dynamic Sound Sources , author=. arXiv preprint arXiv:2606.14141 , year=

  71. [79]

    arXiv preprint arXiv:2509.26140 , year=

    OWL: Geometry-Aware Spatial Reasoning for Audio Large Language Models , author=. arXiv preprint arXiv:2509.26140 , year=

  72. [80]

    International Conference on Machine Learning , pages=

    BAT: Learning to Reason about Spatial Sounds with Large Language Models , author=. International Conference on Machine Learning , pages=. 2024 , organization=

    2024

  73. [81]

    Advances in Neural Information Processing Systems , volume=

    Pengi: An audio language model for audio tasks , author=. Advances in Neural Information Processing Systems , volume=

  74. [82]

    International Conference on Learning Representations , volume=

    Listen, think, and understand , author=. International Conference on Learning Representations , volume=

  75. [83]

    International Conference on Learning Representations , volume=

    Salmonn: Towards generic hearing abilities for large language models , author=. International Conference on Learning Representations , volume=

  76. [84]

    International Conference on Machine Learning , pages=

    Audio Flamingo: A Novel Audio Language Model with Few-Shot Learning and Dialogue Abilities , author=. International Conference on Machine Learning , pages=. 2024 , organization=

    2024

  77. [85]

    Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

    Egocentric audio-visual object localization , author=. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

  78. [86]

    2026 , howpublished=

    2026

  79. [87]

    2025 , howpublished=

    2025

  80. [88]

    ArXiv , year=

    Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities , author=. ArXiv , year=

  81. [89]

    ArXiv , year=

    Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAs , author=. ArXiv , year=

  82. [90]

    ArXiv , year=

    Qwen3-Omni Technical Report , author=. ArXiv , year=

  83. [91]

    Advances in Neural Information Processing Systems , volume=

    Soundspaces 2.0: A simulation platform for visual-acoustic learning , author=. Advances in Neural Information Processing Systems , volume=

  84. [92]

    Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

    Mix and localize: Localizing sound sources in mixtures , author=. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

  85. [93]

    European Conference on Computer Vision , pages=

    Localizing visual sounds the easy way , author=. European Conference on Computer Vision , pages=. 2022 , organization=

    2022

  86. [94]

    Can Large Language Models Understand Spatial Audio? , author=. Proc. Interspeech 2024 , pages=

    2024

  87. [95]

    arXiv preprint arXiv:2511.06606 , year=

    SPUR: A Plug-and-Play Framework for Integrating Spatial Audio Understanding and Reasoning into Large Audio-Language Models , author=. arXiv preprint arXiv:2511.06606 , year=

  88. [96]

    IEEE Open Journal of Signal Processing , year=

    Sci-Phi: A Large Language Model Spatial Audio Descriptor , author=. IEEE Open Journal of Signal Processing , year=

  89. [97]

    arXiv preprint arXiv:2606.10738 , year=

    Spatial-Omni: Spatial Audio Understanding Integration in Multimodal LLMs via FOA Encoding , author=. arXiv preprint arXiv:2606.10738 , year=

    arXiv

  90. [98]

    arXiv preprint arXiv:2509.14666 , year=

    Spatial Audio Motion Understanding and Reasoning , author=. arXiv preprint arXiv:2509.14666 , year=

  91. [99]

    arXiv preprint arXiv:2602.16334 , year=

    Spatial audio question answering and reasoning on dynamic source movements , author=. arXiv preprint arXiv:2602.16334 , year=

  92. [100]

    arXiv preprint arXiv:2606.14141 , year =

    Spatio-Temporal Audio Language Modeling for Dynamic Sound Sources , author =. arXiv preprint arXiv:2606.14141 , year =. doi:10.48550/arXiv.2606.14141 , url =

    doi

  93. [101]

    2024 , volume =

    Zheng, Zhisheng and Peng, Puyuan and Ma, Ziyang and Chen, Xie and Choi, Eunsol and Harwath, David , booktitle =. 2024 , volume =

    2024

  94. [102]

    2026 , url =

    Biswas, Subrata and Khan, Mohammad Nur Hossain and Islam, Bashima , booktitle =. 2026 , url =

    2026

  95. [103]

    2017 International Conference on 3D Vision (3DV) , pages =

    Chang, Angel and Dai, Angela and Funkhouser, Thomas and Halber, Maciej and Nie. 2017 International Conference on 3D Vision (3DV) , pages =. 2017 , publisher =. doi:10.1109/3DV.2017.00081 , url =

    2017

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.