Pith. sign in

REVIEW 3 major objections 5 minor 46 references

Integrating Spatial and Semantic Embeddings for Stereo Sound Event Localization in Videos

T0 review · 3 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Fusing language-aligned audio and visual embeddings through a Cross-Modal Conformer lifts stereo sound event localization above challenge baselines, with the full recipe ranking second in the 2025 challenge.

desk verdict Solid DCASE system paper with a real second-place result; the internal evidence for semantic-embedding gains is weak due to test-split model selection and no error bars. read the letter →

arxiv 2509.06598 v1 pith:6NOWCFC2 submitted 2025-09-08 eess.AS cs.AIcs.LGeess.IVeess.SP

Davide Berghi , Philip J. B. Jackson This is my paper · ORCID
classification eess.AScs.AIcs.LGeess.IVeess.SP
keywords stereosoundeventlocalizationanddetectionaudio-visualfusionCross-ModalConformerCLAPaudioembeddingsOWL-ViTvisualsyntheticdatapre-trainingsourcedistanceestimationon/off-screenclassification
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that stereo sound event localization and detection in ordinary videos can be improved by injecting semantic information from language-aligned models into a conventional audio localization backbone. The authors add frozen CLAP audio embeddings and frozen OWL-ViT visual embeddings to a CNN-Conformer SELD encoder, merging them through a new Cross-Modal Conformer block that uses cross-attention to let one modality query another. On the DCASE 2025 stereo SELD development set, each added semantic stream raises the F1 score, and the full system, trained on large synthetic audio and audio-visual corpora with left-right channel swapping, keypoint-based visual post-processing, and a four-model ensemble, reaches 48.0% F1 and ranks second in the challenge's Track B. If true, the result shows that pretrained, language-aligned embeddings are a practical way to give SELD systems semantic understanding without training large multimodal models from scratch.

What carries the argument

The central mechanism is the Cross-Modal Conformer (CMC), a modified Conformer block that fuses two modality streams. In a standard Conformer, the same stream passes through feed-forward, multi-head self-attention, and convolutional modules; the CMC instead runs two feed-forward layers in parallel on two modalities, called Alpha and Beta, and replaces self-attention with multi-head cross-attention in which queries come from Alpha and keys and values come from Beta. The paper uses one CMC to fuse the CNN-Conformer SELD encoder's embeddings with frozen CLAP audio embeddings, and a second, two-layer CMC to fuse the resulting audio representation with per-patch OWL-ViT visual tokens, linearly projected from 768 to 512 dimensions. Letting the audio stream repeatedly query semantic information from each pretrained encoder while keeping those encoders frozen is what carries the multimodal fusion. A final feed-forward module maps the fused representation to multi-ACCDDOA vectors for up to three simultaneous tracks, trained with class-wise ADPIT loss.

What would settle it

Re-run the full recipe with the same architecture and compute but replace the synthetic pre-training corpora with an equal number of real clips from the development set; if the F1 gap between that model and the reported 48.0% is small, then synthetic data realism is not carrying the result, and the paper's attribution of its gains to pre-training would be undercut.

Watch

Extended reading notes

Core claim

The central claim is that a SELD system for stereo audio, where only azimuth in the range of ±90°, distance, and on/off-screen status are predicted, benefits from semantic embeddings extracted by frozen language-aligned encoders. The paper's ablation shows the SELD encoder alone reaches 34.6% F1, adding CLAP raises it to 35.5%, and adding OWL-ViT raises it to 36.1%, against a 26.8% audio-visual baseline. With the full recipe, including ILD and stpACC input features, pre-training on 5,000 synthetic FOA clips and 2,000 synthetic audio-visual clips, left-right channel swapping, a weighted on/off-screen loss, human keypoint post-processing, and an ensemble of four systems, the model reaches 48.0% F1 on the development set and placed second in the DCASE 2025 Challenge Task 3 (Track B). The paper reads this as evidence that semantically rich, language-aligned embeddings and a fusion module tuned for cross-modal queries carry information that spatial-temporal audio features alone do not provide.

Load-bearing premise

The central assumption is that the synthetic audio and audio-visual clips used for pre-training are realistic enough that the large gains they produce on the development set also transfer to the challenge evaluation data.

Editorial extensions

If this is right

  • The two-step fusion recipe means pretrained language-aligned encoders can be added to an existing SELD backbone without fine-tuning them, so semantic SELD is achievable with only a small trainable fusion module.
  • Because the stereo channels carry no inter-channel time differences, the paper demonstrates that ILD for direction plus stpACC for distance, combined with semantic context, is a viable feature set for stereo 3D SELD.
  • Pre-training on the large synthetic audio and audio-visual corpora is the largest single driver of the reported gains, and using synthetic-domain input normalization generalizes better than using development-set statistics.
  • Audio-only variants with CLAP embeddings match or slightly beat the audio-visual variants on F1 and DOAE, while the visual stream is what enables on/off-screen classification.
  • The ensemble rule, which requires agreement from at least two systems except for Bell and Knock classes, yields further consistent gains in F1, DOAE, RDE, and on/off accuracy.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • One step the paper does not take is replacing the 1 fps, temporally pooled OWL-ViT stream with a video-language model that keeps motion; since temporal dynamics are otherwise left to the audio encoder, a temporally aware visual encoder is a natural testable extension that could improve on/off-screen accuracy.
  • The finding that synthetic-domain normalization statistics generalize better than fine-tuning-domain statistics suggests a broader transfer-learning recipe: keep the preprocessing distribution fixed across pretraining and fine-tuning.
  • Because CLAP alone supplies most of the semantic gain, an inexpensive practical consequence is that stereo or even mono audio-only SELD could benefit from language-aligned audio embeddings, potentially reducing reliance on microphone arrays while retaining competitive detection.
  • The soft cross-fade and indoor background selection address a specific synthetic-to-real visual gap; a direct next experiment is to measure sensitivity to the background distribution by swapping indoor for outdoor synthetic visuals and observing the change in on/off-screen accuracy.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper addresses stereo sound event localization and detection with distance estimation (3D SELD) in regular video content, in the setting of the DCASE 2025 Challenge Task 3 Track B. The authors propose an audio-visual architecture that fuses conventional SELD audio embeddings from a CNN-Conformer encoder with semantic embeddings from CLAP (audio) and OWL-ViT (visual), using a modified Conformer module called the Cross-Modal Conformer (CMC). The training recipe includes pre-training on large synthetic audio and audio-visual corpora (audio 5k and audio-visual 2k), ILD and stpACC input features, on/off-screen weighted loss, YOLOv11-Pose visual post-processing, and majority-vote ensembling. Ablation results in Table 1 and full-system results in Table 2 are reported on the DCASE development set, and the full system is stated to have achieved second rank in the challenge Track B. The central claim is that integrating CLAP and OWL-ViT embeddings improves stereo SELD performance and that the full training recipe produces a top-tier system.

Significance. If the reported results hold, the paper describes a strong, practically relevant system for audio-visual 3D SELD and demonstrates a sensible way to inject language-aligned semantic embeddings into a classical SELD pipeline. The second-place rank in the externally evaluated DCASE 2025 Challenge Task 3 Track B is credible evidence that the full recipe is competitive. The paper also contributes a detailed description of synthetic data curation, including a publicly listed data release, and a careful engineering study of features, augmentation, post-processing, and ensembling. However, the internal attribution of the gains to CLAP and OWL-ViT is weakened by the model-selection procedure, by the absence of repeated runs or error bars, and by the mismatch between the ablation setup and the full-system setup. The challenge rank validates the whole pipeline but does not isolate the contribution of the semantic embeddings, which is the paper's central architectural claim.

major comments (3)
  1. [Section 3.1, Tables 1 and 2] Section 3.1 states that after fine-tuning on the real development set, the best model is selected via F1 score on the test split, and Tables 1 and 2 report all metrics on this same development-test split. Selecting checkpoints on the evaluation split and then reporting numbers from that split makes the reported increments — for example, SE to SE+CLAP (+0.9 F1), SE+CLAP to SE+CLAP+OV (+0.6 F1), and the much larger Table 2 gains over Table 1 — optimistically biased estimates. No repeated runs or confidence intervals are provided, so the magnitude of the selection bias cannot be assessed. The external challenge ranking supports the competitiveness of the full pipeline but does not support the specific attribution of these improvements to CLAP and OWL-ViT. Please report results on a held-out validation split or provide multiple-seed results with error bars.
  2. [Section 3.2.1 vs. Section 3.2.2 and Tables 1 and 2] The ablation study in Table 1 uses a different training setup from the full method in Table 2: the ablation uses only log-mel spectrograms, skips the audio 5k and audio-visual 2k pre-training stages, trains on 15,000 synthetic audio-visual clips, and does not use the weighted on/off-screen loss, visual post-processing, or ensembling. The full method adds ILD and stpACC features, large-scale pre-training, and several engineering refinements. Consequently, the large performance gap between Tables 1 and 2 cannot be attributed specifically to synthetic pre-training or to the semantic embeddings; the contributions of the components are confounded. If the central claim is that the Cross-Modal Conformer with CLAP and OWL-ViT helps, the relevant evidence is the Table 1 ablation, but that evidence is subject to the model-selection problem noted above. I recommend either adding a full-recipe ablation that isolates CLAP/OWL-ViT under the final training setup or explicitly narrowing the causal claims to the Table 1 setting.
  3. [Section 3.3.2] The claim that using synthetic-data normalization statistics for system (2) improves generalization is supported by challenge evaluation numbers stated in the text (29.4% vs. 42.5% F1) but not included in any table or referenced to a public leaderboard page. Since the development-set numbers for systems (1) and (2) are nearly identical (45.7 vs. 46.0 F1), the evaluation-subset comparison is the only evidence for this claim. Please tabulate the official evaluation-subset scores for all submitted systems or provide a direct citation to the published challenge results, so this claim can be verified.
minor comments (5)
  1. [Section 2.1.3] The non-linear spatial re-framing transformation is described only verbally; please provide a formula or a reference, since it is a design choice that reportedly affects performance.
  2. [Section 2.3] The relationship between the 2,000 one-minute FOA audio-visual clips and the "+60,000 audio-visual clips" is not spelled out; clarify how the 5-second segmentation, rotations, and augmentation produce this count, as was done for the audio 5k dataset.
  3. [Section 3.3.2] The text states that the visual post-processing step affects on/off-screen accuracy and F1/on but leaves other metrics unchanged; Tables 2 and 2.1 are consistent with this, but the identical Acc values across some rows (e.g., systems (3) and (3.1)) make the claimed effect hard to see. A short explanation of why Acc is unchanged while F1/on improves would help.
  4. [General] No training code is provided, although a data link is given; given the complexity of the recipe, releasing code or a more detailed configuration would improve reproducibility.
  5. [Figure 1] The notation "Alpha" and "Beta" in the figure caption is clear in context, but the mathematical symbols α and β are not defined in the text; please define them in Section 2.1.2.

Circularity Check

1 steps flagged · score 6.0 of 10

Development-test checkpoints are selected on the same split used to report ablation gains, so the CLAP/OWL-ViT contribution numbers are selected maxima; the external second-place rank is not affected.

  1. fitted input called prediction [Sec. 3.1 (Implementation Details), applied to Tables 1 and 2]
    "fine-tuned for another 80 epochs on the real development set, selecting the best model via F1 score on the test split."

    Model selection uses the same development-test F1 that is later reported as the result. Tables 1 and 2 present F1, F1o, DOAE, RDE, and Acc on this split; because the best checkpoint/epoch for each system was chosen by maximizing F1 on this split, the reported values are selected maxima, not held-out predictions. The ablation deltas (+0.9 CLAP, +0.6 OWL-ViT) are differences of such maxima and can be inflated by selection noise. The paper provides no multiple seeds or error bars to correct for this, so the claimed individual contributions of the language-aligned models are not independently evidenced by the development-set comparison.

full rationale

The paper's strongest external evidence is the DCASE 2025 Task 3 Track B second-place ranking, which is held-out and therefore not circular; it shows the full pipeline is competitive. The internal derivation chain, however, has one load-bearing evaluative circularity: Sec. 3.1 states that after fine-tuning the best model is selected by F1 on the development test split, and the ablation and proposed-system tables report metrics on exactly this split. The contributions attributed to CLAP and OWL-ViT (e.g., F1 +0.9 and +0.6 in Table 1) are differences of values that were used as selection criteria, so they are selected maxima rather than independent estimates. This is not a self-citation chain: the CMC is an acknowledged adaptation of the external Conformer work, stpACC is described on its own, and no uniqueness theorem is imported from the authors' prior papers. But the abstract's claim that the ablation assesses the individual contributions of the language-aligned models is not supported as stated, because the development-test numbers have been fit by the selection procedure. The external rank cannot isolate the semantic contribution because the full system differs from the ablation in pre-training, input features, loss weighting, visual post-processing, and ensembling. Thus the paper is partially circular in its internal evaluation, though not in its external benchmark evaluation.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claim depends on standard deep learning pipeline choices: domain assumptions about the data (stereo derived from FOA, synthetic-to-real transfer) and about the usefulness of language-aligned embeddings. The free parameters listed are manually chosen hyperparameters that influence the on/off-screen and ensemble metrics; none are fit to a scientific law. No new entities are invented.

free parameters (3)
  • on-screen BCE loss weight = 4.0
    Multiplier applied to the binary cross-entropy term when the ground truth label is on-screen (Sec. 3.2.2). Chosen by hand; affects on/off-screen metrics in system (4).
  • keypoint angular threshold = 20 degrees
    Events whose DOA falls within 20 degrees of a detected keypoint are marked on-screen (Sec. 3.2.2). Manually set; determines the post-processing gain in F1o.
  • ensemble agreement threshold = 2 systems within 20 degrees
    An event is active if at least two systems agree and their DOAs are within 20 degrees (Sec. 3.2.2). Manually set; shapes the ensemble result in system (5).
assumptions (4)
  • domain assumption Stereo channels are arithmetic downmixes of FOA and therefore contain no inter-channel time differences; ILD is the only reliable spatial cue.
    Sec. 2.2 states this as the basis for using ILD rather than ITD features. If the dataset's stereo channels had real phase differences, the feature design would be suboptimal.
  • domain assumption OWL-ViT's per-token embeddings retain both spatial and semantic information useful for sound localization and on/off-screen reasoning.
    Sec. 2.1.3 argues that preserving tokens instead of channel-pooling preserves spatial resolution. This is a stated belief with no direct probe.
  • domain assumption Synthetic audio-visual data generated by SpatialScaper and SELDVisualSynth transfers to real recorded content.
    Sec. 2.3; the large improvement from pre-training in Table 2 depends on this transfer.
  • standard math ADPIT loss and the multi-ACCDOA output representation are accepted as correct for this task.
    The paper builds on [22]-[25] without deriving or questioning the loss formalism; this is standard in SELD.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Integrating Spatial and Semantic Embeddings for Stereo Sound Event Localization in Videos." pith.science (2026). https://pith.science/paper/6NOWCFC2

@misc{pith2026250906598,
  author       = {Pith},
  title        = {Pith review of: Integrating Spatial and Semantic Embeddings for Stereo Sound Event Localization in Videos},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6NOWCFC2}},
  note         = {Machine review of arXiv:2509.06598}
}
read the original abstract

In this study, we address the multimodal task of stereo sound event localization and detection with source distance estimation (3D SELD) in regular video content. 3D SELD is a complex task that combines temporal event classification with spatial localization, requiring reasoning across spatial, temporal, and semantic dimensions. The last is arguably the most challenging to model. Traditional SELD approaches typically rely on multichannel input, limiting their capacity to benefit from large-scale pre-training due to data constraints. To overcome this, we enhance a standard SELD architecture with semantic information by integrating pre-trained, contrastive language-aligned models: CLAP for audio and OWL-ViT for visual inputs. These embeddings are incorporated into a modified Conformer module tailored for multimodal fusion, which we refer to as the Cross-Modal Conformer. We perform an ablation study on the development set of the DCASE2025 Task3 Stereo SELD Dataset to assess the individual contributions of the language-aligned models and benchmark against the DCASE Task 3 baseline systems. Additionally, we detail the curation process of large synthetic audio and audio-visual datasets used for model pre-training. These datasets were further expanded through left-right channel swapping augmentation. Our approach, combining extensive pre-training, model ensembling, and visual post-processing, achieved second rank in the DCASE 2025 Challenge Task 3 (Track B), underscoring the effectiveness of our method. Future work will explore the modality-specific contributions and architectural refinements.

Figures

Figures reproduced from arXiv: 2509.06598 by the authors.

Figure 1
Figure 1. Proposed audio-visual model (left). Blue blocks are frozen, [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Examples of synthetic scenes (clockwise from top left) in [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

46 extracted references · 30 canonical work pages

  1. [1]

    Sound event localization and detection of overlapping sources using convolutional recurrent neural networks,

    S. Adavanne et al., “Sound event localization and detection of overlapping sources using convolutional recurrent neural networks,”IEEE J. of Selected Topics in Sig. Proc., vol. 13, pp. 34–48, 2019

    work page 2019

  2. [2]

    Sound event detection using spatial features and convolutional recurrent neural network,

    S. Adavanne et al., “Sound event detection using spatial features and convolutional recurrent neural network,” inICASSP, 2017

    2017

  3. [3]

    Direction of arrival estimation for multiple sound sources using convolutional recurrent neural network,

    S. Adavanne, A. Politis, and T. Virtanen, “Direction of arrival estimation for multiple sound sources using convolutional recurrent neural network,” inEUSIPCO, 2018, pp. 1462–1466

    work page 2018

  4. [4]

    Audio-visual cross-attention network for robotic speaker tracking,

    X. Qian et al., “Audio-visual cross-attention network for robotic speaker tracking,”IEEE/ACM Trans. Audio, Speech & Lang. Proc., vol. 31, pp. 550–562, 2023

    work page 2023

  5. [5]

    ForecasterFlexOBM: A multi-view audio-visual dataset for flexible object-based media production,

    D. Berghi et al., “ForecasterFlexOBM: A multi-view audio-visual dataset for flexible object-based media production,” inICME, 2024

    work page 2024

  6. [6]

    A dataset of reverberant spatial sound scenes with moving sources for sound event localization and detection,

    A. Politis et al., “A dataset of reverberant spatial sound scenes with moving sources for sound event localization and detection,” inDCASE Workshop, 2020

    2020

  7. [7]

    A dataset of dynamic reverberant sound scenes with directional interferers for sound event localization and detection,

    A. Politis et al., “A dataset of dynamic reverberant sound scenes with directional interferers for sound event localization and detection,” in DCASE Workshop, 2021

    2021

  8. [8]

    STARSS23: An audio-visual dataset of spatial recordings of real scenes with spatiotemporal annotations of sound events,

    K. Shimada et al., “STARSS23: An audio-visual dataset of spatial recordings of real scenes with spatiotemporal annotations of sound events,” inNeurIPS, 2023

    work page 2023

Show all 46 references
  1. [9]

    Baseline models and evaluation of sound event localization and detection with distance estimation in DCASE 2024 Challenge,

    D. Diaz-Guerra et al., “Baseline models and evaluation of sound event localization and detection with distance estimation in DCASE 2024 Challenge,” inDCASE Workshop, 2024, pp. 41–45

    2024

  2. [10]

    Language models are few-shot learners,

    T. Brown et al., “Language models are few-shot learners,” inNeurIPS, 2020

    2020

  3. [11]

    Learning transferable visual models from natural language supervision,

    A. Radford et al., “Learning transferable visual models from natural language supervision,” inInt. Conf. on Machine Learning, 2021

    2021

  4. [12]

    AudioGPT: understanding and generating speech, music, sound, and talking head,

    R. Huang et al., “AudioGPT: understanding and generating speech, music, sound, and talking head,” inAAAI Conf. on Artificial Intelligence, 2024

    2024

  5. [13]

    Spatial Scaper: A library to simulate and augment soundscapes for sound event localization and detection in realistic rooms,

    I. R. Roman et al., “Spatial Scaper: A library to simulate and augment soundscapes for sound event localization and detection in realistic rooms,” inICASSP, 2024

    2024

  6. [14]

    Generating diverse audio-visual 360 ◦ soundscapes for sound event localization and detection,

    A. S. Roman et al., “Generating diverse audio-visual 360 ◦ soundscapes for sound event localization and detection,”ArXiv, vol. abs/2504.02988, 2025

    2025 arXiv

  7. [15]

    Large-scale contrastive language-audio pretraining with feature fusion and keyword-to-caption augmentation,

    Y . Wu et al., “Large-scale contrastive language-audio pretraining with feature fusion and keyword-to-caption augmentation,” inICASSP, 2023

    2023

  8. [16]

    Simple open-vocabulary object detection,

    M. Minderer et al., “Simple open-vocabulary object detection,” inECCV, 2022

    2022

  9. [17]

    A four-stage data augmentation approach to ResNet- Conformer based acoustic modeling for sound event localization and detection,

    Q. Wang et al., “A four-stage data augmentation approach to ResNet- Conformer based acoustic modeling for sound event localization and detection,”IEEE/ACM TASLP, vol. 31, pp. 1251–1264, 2023

    2023

  10. [18]

    Fusion of audio and visual embeddings for sound event localization and detection,

    D. Berghi et al., “Fusion of audio and visual embeddings for sound event localization and detection,” inICASSP, 2024

    2024

  11. [19]

    ResNet-Conformer network using multi-scale channel attention for sound event localization and detection in real scenes,

    L. Xue et al., “ResNet-Conformer network using multi-scale channel attention for sound event localization and detection in real scenes,” in Int. Conf. on Wireless Comms. & Sig. Proc., 2023

    2023

  12. [20]

    Conformer: Convolution-augmented transformer for speech recognition,

    A. Gulati et al., “Conformer: Convolution-augmented transformer for speech recognition,” inInterspeech, 2020, pp. 5036–5040

    2020

  13. [21]

    Reverberation-based features for sound event localization and detection with distance estimation,

    D. Berghi and P. J. B. Jackson, “Reverberation-based features for sound event localization and detection with distance estimation,”ArXiv, vol. abs/2504.08644, 2025

    2025 arXiv

  14. [22]

    Sound event detection and localization with distance estimation,

    D. A. Krause et al., “Sound event detection and localization with distance estimation,” inEUSIPCO, 2024, pp. 286–290

    2024

  15. [23]

    Multi-ACCDOA: Localizing and detecting overlapping sounds from the same class with auxiliary duplicating permutation invariant training,

    K. Shimada et al., “Multi-ACCDOA: Localizing and detecting overlapping sounds from the same class with auxiliary duplicating permutation invariant training,” inICASSP, 2022, pp. 316–320

    2022

  16. [24]

    Event-independent network for polyphonic sound event localization and detection,

    Y . Cao et al., “Event-independent network for polyphonic sound event localization and detection,” inDCASE Workshop, 2020

    2020

  17. [25]

    An improved event-independent network for polyphonic sound event localization and detection,

    Y . Cao et al., “An improved event-independent network for polyphonic sound event localization and detection,” inICASSP, 2021

    2021

  18. [26]

    Batch normalization: Accelerating deep network training by reducing internal covariate shift,

    S. Ioffe and C. Szegedy, “Batch normalization: Accelerating deep network training by reducing internal covariate shift,” inInternational Conference on Machine Learning, vol. 37, 2015, pp. 448–456

    2015

  19. [27]

    Leveraging reverberation and visual depth cues for sound event localization and detection with distance estimation,

    D. Berghi and P. J. B. Jackson, “Leveraging reverberation and visual depth cues for sound event localization and detection with distance estimation,” inTechical Report of DCASE Challenge, 2024

    2024

  20. [28]

    Deep residual learning for image recognition,

    K. He et al., “Deep residual learning for image recognition,” inIEEE Conf. on Comp. Vis. & Pattern Recogn. (CVPR), 2016, pp. 770–778

    2016

  21. [29]

    MV ANet: Multi-stage video attention network for sound event localization and detection with source distance estimation,

    H. Hong et al., “MV ANet: Multi-stage video attention network for sound event localization and detection with source distance estimation,”ArXiv, vol. abs/2411.14153, 2024

    2024 arXiv

  22. [30]

    First order ambisonics domain spatial augmentation for DNN-based direction of arrival estimation,

    L. Mazzon et al., “First order ambisonics domain spatial augmentation for DNN-based direction of arrival estimation,” inDCASE Workshop, 2019

    2019

  23. [31]

    Two vs. four-channel sound event localization and detection,

    J. Wilkins et al., “Two vs. four-channel sound event localization and detection,” inDCASE Workshop, 2023, pp. 216–220

    2023

  24. [32]

    SA VGBench: Benchmarking spatially aligned audio- video generation,

    K. Shimada et al., “SA VGBench: Benchmarking spatially aligned audio- video generation,”ArXiv, vol. abs/2412.13462, 2025

    2025

  25. [33]

    FSD50K: an open dataset of human-labeled sound events,

    E. Fonseca et al., “FSD50K: an open dataset of human-labeled sound events,”IEEE/ACM TASLP, vol. 30, pp. 829–852, 2022

    2022

  26. [34]

    METU SPARG Eigenmike em32 Acoustic Impulse Response Dataset v0.1.0,

    O. Olgun and H. Hacihabiboglu, “METU SPARG Eigenmike em32 Acoustic Impulse Response Dataset v0.1.0,” Apr. 2019. [Online]. Available: https://doi.org/10.5281/zenodo.2635758

    2019 doi

  27. [35]

    Dataset of spatial room impulse responses in a variable acoustics room for six degrees-of-freedom rendering and analysis,

    T. McKenzie, L. McCormack, and C. Hold, “Dataset of spatial room impulse responses in a variable acoustics room for six degrees-of-freedom rendering and analysis,”ArXiv, vol. 2111.11882, 2021

    2021 arXiv

  28. [36]

    FMA: A dataset for music analysis,

    M. Defferrard, K. Benzi, P. Vandergheynst, and X. Bresson, “FMA: A dataset for music analysis,”ArXiv, vol. abs/1612.01840, 2016

    2016 arXiv

  29. [37]

    A dataset of higher-order ambisonic room impulse responses and 3D models measured in a room with varying furniture,

    G. G ¨otz et al., “A dataset of higher-order ambisonic room impulse responses and 3D models measured in a room with varying furniture,” inImmersive and 3D Audio (I3DA), 2021

    2021

  30. [38]

    Room impulse response dataset of a recording studio with variable wall paneling measured using a 32-channel spherical microphone array and a B-format microphone array,

    G. Chesworth, A. Bastine, and T. Abhayapala, “Room impulse response dataset of a recording studio with variable wall paneling measured using a 32-channel spherical microphone array and a B-format microphone array,”Applied Sciences, vol. 14, no. 5, p. 2095, 2024

    2024

  31. [39]

    Data set: Eigenmike-DRIRs, KEMAR 45BA- BRIRs, RIRs and 360° pictures captured at five positions of a small conference room,

    C. Schneiderwind et al., “Data set: Eigenmike-DRIRs, KEMAR 45BA- BRIRs, RIRs and 360° pictures captured at five positions of a small conference room,” inProc. 45th DAGA, 2019

    2019

  32. [40]

    Flickr30k entities: Collecting region-to-phrase correspondences for richer image-to-sentence models,

    B. A. Plummer et al., “Flickr30k entities: Collecting region-to-phrase correspondences for richer image-to-sentence models,” inICCV, 2015

    2015

  33. [41]

    Robust and adaptive door operation with a mobile robot,

    M. Arduengo et al., “Robust and adaptive door operation with a mobile robot,”Intelligent Service Robotics, p. 409–425, 2021

    2021

  34. [42]

    NitroFusion: High-fidelity single-step diffusion through dynamic adversarial training,

    D.-Y . Chen et al., “NitroFusion: High-fidelity single-step diffusion through dynamic adversarial training,”ArXiv, vol. abs/2412.02030, 2024

    2024 arXiv

  35. [43]

    360-Indoor: Towards learning real-world objects in 360° indoor equirectangular images,

    S.-H. Chou et al., “360-Indoor: Towards learning real-world objects in 360° indoor equirectangular images,” inIEEE Winter Conf. on Applications of Comp. Vis., 2020

    2020

  36. [44]

    Exploring audio-visual information fusion for sound event localization and detection in low-resource realistic scenarios,

    Y . Jiang et al., “Exploring audio-visual information fusion for sound event localization and detection in low-resource realistic scenarios,” in ICME, 2024, pp. 1–6

    2024

  37. [45]

    Enhanced sound event localization and detection in real 360-degree audio-visual soundscapes,

    A. S. Roman et al., “Enhanced sound event localization and detection in real 360-degree audio-visual soundscapes,”ArXiv, vol. abs/2401.17129, 2024

    2024 arXiv

  38. [46]

    Ultralytics YOLO,

    G. Jocher, Q. Jing, and A. Chaurasia, “Ultralytics YOLO,” Accessed Jun. 21, 2025, 2023, https://github.com/ultralytics/ultralytics

    2025

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.