Sound source detection, localization and classification using consecutive ensemble of CRNN models

Mateusz Lewandowski; S{\l}awomir Kapka

REVIEW 1 major objections 4 minor 26 references

Sound source detection, localization and classification using consecutive ensemble of CRNN models

T0 review · 1 major / 4 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that sound-event localization decomposes into four sequential single-output CRNN subtasks, and that the resulting consecutive ensemble ranked first in the 2019 challenge: error rate 0.08, F-score 94.7, DOA error 3.7…

desk verdict A rank-1 DCASE 2019 system with a genuine decomposition idea, undermined only by an STFT hop-length typo that blocks exact replication. read the letter →

arxiv 1908.00766 v2 pith:HT5PTUK3 submitted 2019-08-02 eess.AS cs.LGcs.SDstat.ML

S{l}awomir Kapka , Mateusz Lewandowski This is my paper · ORCID

classification eess.AScs.LGcs.SDstat.ML

keywords soundeventlocalizationanddetectionCRNNconsecutiveensembledirectionofarrivalambisonicsnumberactivesourcesDCASE2019task3polyphonicaudio

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to show that sound-event localization and detection—deciding what sounds are present, when they start and stop, and where they come from—can be broken into simpler subproblems solved one after another. Instead of one network producing all outputs at once, four small SELDnet-like CRNNs run in sequence: one counts active sources, one finds the direction of a lone source, one finds the second source's direction given the first, and one assigns class labels. On the 2019 challenge's official evaluation set this consecutive ensemble achieves error rate 0.08, F-score 94.7, DOA error 3.7 degrees, and frame recall 96.8, which ranked first overall. The broader point is that task decomposition with explicit intermediate decisions can outperform a monolithic end-to-end network on this benchmark.

What carries the argument

The load-bearing mechanism is the consecutive ensemble: four single-output CRNN models chained so that each stage's prediction becomes the next stage's conditioning. The noas model emits a per-frame source count from 256-frame amplitude spectrograms; a postprocessor forces legitimate count trajectories (start and end at zero, step size at most one), and event onsets and offset candidates are derived from count transitions. The doa1 and doa2 models predict Cartesian xyz directions, with doa2 receiving the associated event's direction as an extra input; the class model uses soft voting across frames and, for fully overlapping events, picks the most probable class different from the associated event's class. Direction outputs are rounded to a 10-degree grid, which the algorithm exploits to decide whether two consecutive single-source chunks belong to the same event.

What would settle it

Feed a one-minute recording through an STFT with a 0.4-second Hann window and a 0.2-second hop and count the output frames: the result is about 300, not the 3000 vectors the architecture expects, which settles whether the printed description can reproduce the reported scores.

Watch

Extended reading notes

Core claim

The authors propose that sound-event localization and detection should be solved by decomposition rather than by one monolithic network. Their system consists of four SELDnet-like CRNNs, each with a single output, arranged as a consecutive ensemble: the first predicts the number of active sources in each frame; the second predicts the direction of arrival of a lone source; the third predicts the direction of a second source conditioned on the associated event's direction; the fourth performs multi-label classification. The counting output is postprocessed so that each recording starts and ends silent and the count changes by at most one per frame; onsets and candidate offsets are read from count transitions, then disambiguated by comparing averaged direction estimates across chunks of constant count. On the official evaluation set, the system achieves error rate 0.08, F-score 94.7, DOA error 3.7 degrees, and frame recall 96.8, the best overall score among the submitted systems.

Load-bearing premise

The load-bearing premise is that the printed feature-extraction settings match the submitted runs, but they do not as written: a 0.2-second STFT hop on a one-minute recording yields about 300 frames, not the 3000-frame inputs the ensemble is described to consume.

Editorial extensions

If this is right

Systems that adopt this decomposition can treat source-count estimation as an explicit first stage, and the official numbers suggest that stage is accurate enough to support competitive localization.
Because the system uses only first-order ambisonic input, the results indicate that the four-channel microphone format is not necessary for top performance on this benchmark.
The gap between single-source and overlapping-source results (DOA error 1.28 degrees versus 7.96 degrees, error rate 0.07 versus 0.18) implies the method's strength lies in single-source frames, and overlapping cases remain the harder regime.
The pipeline's sequential nature means count errors propagate: a misclassified source-count frame can shift onset and offset assignments and corrupt both direction and class predictions for that segment.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If the 3000-frame statement reflects the real configuration, the implied STFT hop is close to 0.02 seconds, so correcting the printed 0.2-second value would be the first step toward reproducing the result from the paper alone.
The same decompose-and-chain scheme could be extended to three or more simultaneous sources by adding a doa3 network conditioned on two associated directions; the paper's stated at-most-two assumption marks exactly where that extension begins.
Rounding predicted directions to the nearest 10 degrees means the reported 3.7-degree DOA error partly reflects the grid rather than raw model accuracy, and an off-grid or finer-grid evaluation would separate quantization from estimation error.
A soft version of the source-count stage—for example, a probability distribution over 0, 1, or 2 instead of a hard pick—might reduce cascade failures when the count is ambiguous, at the cost of losing the simple event-segmentation rule.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Colleague,

The short version: this is a first-place DCASE 2019 Task 3 submission that actually earns its rank. The new idea is the consecutive ensemble: four CRNNs that predict source count, single-source DOA, conditional two-source DOA, and class one after another, rather than one network for everything. That decomposition is genuinely different from the two-stage approaches in the same competition, and the official test scores (error rate 0.08, F-score 94.7, DOA error 3.7, frame recall 96.8) are externally evaluated, so the headline result is credible.

The paper does a lot right. The architecture table is detailed, the algorithm is described step by step with a worked example, and the authors are honest about the limitations (max two sources, fixed 10° grid). The development-set results with all four folds are there, and the comparison with other submissions is fair. I have no reason to doubt the numbers.

The soft spot is real and it is the reader's weakest assumption. Section 2 says the STFT uses a 0.2 s hop, but then both Section 2 and Section 4.1 say a one-minute recording yields exactly 3000 frames. With a 0.2 s hop you would get about 300 frames, not 3000. A 0.02 s hop would give about 2981 frames, so even that needs an unspecified truncate/pad step. This is not a minor cosmetic typo: the input sequence lengths in Table 1 (256 and 128) and every model input depend on this frame count. Anyone following the printed hop will build a different feature pipeline and will not reproduce the results. The authors likely meant 0.02 s, but the paper does not say that.

The other caveats are minor. The postprocessing rules (NOAS smoothing, event assignment, class disambiguation) are heuristic and tuned to this challenge; that is normal for challenge papers but it limits how far the lesson generalizes. The paper also does not release code, so the only path to reproduction is the textual description, which is exactly where the typo hurts.

Bottom line: this is a solid system paper with a genuine rank-1 result and a clear methodological contribution. It deserves a serious referee and, with a one-line fix to the STFT parameters, it would be fully reproducible. I would recommend accepting it after that correction.

Referee Report

1 major / 4 minor

Summary. The paper describes a system for DCASE 2019 Task 3 (Sound Event Localization and Detection) on first-order ambisonic recordings. The SELD task is decomposed into four subtasks: estimating the number of active sources (NOAS), estimating the direction of arrival of a single source (doa1), estimating the direction of arrival of a second source given an associated event (doa2), and multi-label classification of sound events. Each subtask is handled by a separate CRNN, and the outputs are combined by a hand-crafted consecutive ensemble that predicts event onsets, offsets, DOAs, and class labels. The system is evaluated on the TAU Spatial Sound Events 2019 Ambisonic dataset, reporting official test scores of error rate 0.08, F-score 94.7, DOA error 3.7 degrees, and frame recall 96.8, ranked first among challenge submissions and outperforming the official baseline.

Significance. If the reported results are taken at face value, the paper demonstrates a successful alternative to monolithic SELDnet: decomposing the task into specialized models and combining them with deterministic rules yields top-ranked performance on a public challenge. The paper's strengths include the externally evaluated official challenge results (not self-reported), per-fold development metrics with comparisons to baseline and other submissions, a detailed architecture table, and an explicit statement of limitations (more than two overlapping sources, coarse DOA grid, error cascading). The paper does not release code, so reproducibility relies entirely on the written specification; the STFT hop-length inconsistency described below is a genuine obstacle to exact replication.

major comments (1)

[Section 2 and Section 4.1] The STFT configuration is internally inconsistent and blocks replication. Section 2 states a hop length of 0.2 s with a 0.4 s Hann window, while Section 2 and Section 4.1 both state that each recording yields exactly 3000 frames. For a 60 s recording, a 0.2 s hop gives about 299 frames, and a 0.02 s hop gives 2981 frames, neither of which equals 3000 without an additional, unspecified truncation or padding step. Because the input shapes in Table 1 (256- and 128-frame subsequences) and the entire consecutive ensemble algorithm in Section 4 operate on frame indices, the actual STFT hop length and the frame-count normalization must be specified for the system to be reproducible. I request that the authors provide the correct hop length and an explicit statement of how every recording is converted to exactly 3000 frames.

minor comments (4)

[Section 5.1] The description of the development protocol is ambiguous: 'we used 2 splits out of 4 for training for every fold' could mean that two of the four splits are training, one is validation, and one is test, but the standard DCASE 2019 task 3 setup uses three training splits per fold. Please state explicitly the split assignment for each fold and clarify the role of the validation split.
[Section 2] Minor language issues: 'If audio is longer than 1 minute' should be 'If an audio recording is longer than 1 minute', and 'If an audio is shorter than 1 minute' should be 'If an audio recording is shorter than 1 minute'.
[Reference [3]] The reference title contains a typo: 'uetection' should be 'detection'.
[Table 1] Consider clarifying the 'Doa2 input' row: the boolean values indicate per-model usage, but the row label is easy to misread as a layer type.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the reported results are genuine external challenge evaluations, and the system's components are not defined in terms of the quantities they predict.

full rationale

The paper reports results from the DCASE 2019 Task 3 evaluation, comparing against other submissions and the challenge baseline. Development-fold results (Table 3) and official evaluation results (Table 5) are measured on held-out data using the challenge's scoring software, so the reported error rate, F-score, DOA error, and frame recall are genuine predictions rather than fitted or renamed inputs. The four CRNN models are described as independent sub-task predictors (NOAS, DOA1, DOA2, class), and the consecutive ensemble is an explicit inference procedure that combines their outputs; it is not equivalent to any single training target or to the input features by construction. The architecture is based on external SELDnet work [2,3], not on a self-citation chain, and no uniqueness claim or ansatz is imported from the authors' prior work. There is a reproducibility issue in the feature description: the 0.2 s STFT hop would yield roughly 300 frames for a one-minute recording, while Sections 2 and 4.1 state exactly 3000 frames, implying a factor-of-ten inconsistency or a need for truncation/padding. That is an implementation and replication concern, not a circularity concern, because it does not make any predicted quantity equivalent to an input or fitted parameter. The paper also acknowledges limitations (more than two sources, grid resolution), which further indicates the claims are empirically contingent rather than definitionally forced.

Assumptions & free parameters 9 free parameters · 4 assumptions · 0 invented entities

The paper contributes a system architecture and postprocessing algorithm. Its central claim rests on the validity of the task decomposition and on a set of hand-tuned feature extraction and heuristic choices. No new physical entities are introduced.

free parameters (9)

STFT hop length = 0.2 s (text) / likely 0.02 s (consistent with 3000 frames)
Hand-chosen feature-extraction parameter; the stated value contradicts the reported frame count, so the actual value used is unclear.
Input sequence length = 256 frames (noas), 128 frames (doa1/doa2/class)
Chosen by hand; controls the temporal context seen by each network.
Convolutional filter count = 64
Architecture choice inherited from SELDnet.
Recurrent units = 128
Architecture choice inherited from SELDnet.
Number of training epochs = 500 (noas, class), 1000 (doa1, doa2)
Stopping rule chosen by hand; no early stopping or validation-based selection reported.
Learning rate schedule = Adam default, decay 0.95 every 5 epochs for noas/class; constant for doa1/doa2
Hand-chosen schedule.
DOA grid resolution = 10 degrees
Predictions are rounded to the nearest multiple of 10 degrees to match the dataset grid.
NOAS smoothing constraint = max difference of 1 between consecutive frames
Postprocessing rule that forces the predicted source count trajectory to change by at most one per frame; an ad hoc assumption about event onset/offset structure.
Class disambiguation rule = if overlapping event class equals associated class, choose second most probable class
Ad hoc rule to force distinct class labels for simultaneous events.

assumptions (4)

domain assumption The TAU Spatial Sound Events 2019 dataset and its official evaluation metrics provide a valid benchmark for SELD performance.
The paper relies on the DCASE task definition and metrics without questioning them.
domain assumption At most two active sound sources occur simultaneously, and all directions lie on a 10-degree grid.
These are properties of the benchmark that the algorithm explicitly exploits (Sections 1 and 4.1).
ad hoc to paper The SELD task can be decomposed into independent subtasks: source counting, single-source DOA, conditional two-source DOA, and classification.
This is the paper's central modeling choice, invoked throughout Sections 3 and 4. It is not derived from first principles and is specific to the two-source limit.
ad hoc to paper The consecutive postprocessing rules (NOAS smoothing, chunk matching, offset inference) correctly reconstruct event boundaries from frame-level predictions.
Described in Section 4.1; these heuristics are the core of the ensemble and are justified only by the final scores.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Sound source detection, localization and classification using consecutive ensemble of CRNN models." pith.science (2026). https://pith.science/paper/HT5PTUK3

@misc{pith2026190800766,
  author       = {Pith},
  title        = {Pith review of: Sound source detection, localization and classification using consecutive ensemble of CRNN models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HT5PTUK3}},
  note         = {Machine review of arXiv:1908.00766}
}

read the original abstract

In this paper, we describe our method for DCASE2019 task3: Sound Event Localization and Detection (SELD). We use four CRNN SELDnet-like single output models which run in a consecutive manner to recover all possible information of occurring events. We decompose the SELD task into estimating number of active sources, estimating direction of arrival of a single source, estimating direction of arrival of the second source where the direction of the first one is known and a multi-label classification task. We use custom consecutive ensemble to predict events' onset, offset, direction of arrival and class. The proposed approach is evaluated on the TAU Spatial Sound Events 2019 - Ambisonic and it is compared with other participants' submissions.

Figures

Figures reproduced from arXiv: 1908.00766 by the authors.

**Figure 2.** The plot visualising the predicted number of active [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗

**Figure 3.** Given the predicted NOAS from the part of some recording as in the top plot, we deduce that there are 3 events [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

26 extracted references · 20 canonical work pages

[1]

This year, the task 3 from the IEEE AASP Challenge on Detection and Classiﬁcation of Acoustic Scenes and Events (DCASE 2019) [1] concerned the SELD prob- lem

INTRODUCTION Sound Event Localization and Detection (SELD) is a complex task which naturally appears when one wants to develop a sys- tem that possesses spatial awareness of the surrounding world us- ing multi-channel audio signals. This year, the task 3 from the IEEE AASP Challenge on Detection and Classiﬁcation of Acoustic Scenes and Events (DCASE 2019)...

work page 2019
[2]

Sound source detection, localization and classification using consecutive ensemble of CRNN models

FEATURES The DCASE 2019 task 3 provides two formats of the TAU Spa- tial Sound Events 2019 dataset: ﬁrst order ambisonic ( foa) and 4 channels from a microphone array (mic) [3]. In our method we only use the ambisonic format. arXiv:1908.00766v2 [eess.AS] 5 Sep 2019 Detection and Classiﬁcation of Acoustic Scenes and Events 2019 25–26 October 2019, New York...

work page Pith review arXiv 2019
[3]

Each of these models is a copy of a single SELDnet node with just minor adjustments so that it ﬁts to the speciﬁc subtask and for the regular- ization purpose

ARCHITECTURE As mentioned in the introduction, each of the subtasks ( noas, doa1, doa2 and class) has its own SELDnet-like CRNN. Each of these models is a copy of a single SELDnet node with just minor adjustments so that it ﬁts to the speciﬁc subtask and for the regular- ization purpose. Each of these models takes as an input a ﬁxed length subse- quence o...

work page
[4]

This custom binding of our four models allows us to predict the events’ onset, offset, direction of arrival and class

CONSECUTIVE ENSEMBLE In this section, we introduce and describe the idea of the con- secutive ensemble which is the core of our approach. This custom binding of our four models allows us to predict the events’ onset, offset, direction of arrival and class. 4.1. The algorithm We assume that recordings have at most 2 active sound sources at once and the sou...

work page
[5]

We feed the features to the noas network to predict the num- ber of active sources (NOAS) in each frame

work page
[6]

We transform the predicted NOAS so that each recording starts and ends with no sound sources and the difference of NOAS between each frames is no greater than 1

work page
[7]

If NOAS in two consecutive frames increases, then we predict that a new event happened at the second frame

From the predicted NOAS we deduce the number of events, their onsets and the list of possible offsets for each event. If NOAS in two consecutive frames increases, then we predict that a new event happened at the second frame. If in two consecutive frames NOAS decreases, then we append the ﬁrst frame to all events since last time NOAS was 0 as a possible offset

work page
[8]

We extract chunks (intervals of equal NOAS) of audio where the predicted NOAS equals 1 and we feed it to doa1 network

In order to determine which offset corresponds to which event we use the doa1 network. We extract chunks (intervals of equal NOAS) of audio where the predicted NOAS equals 1 and we feed it to doa1 network. For each chunk where NOAS was 1 we predict the average azimuth and elevation, and we round it to the closest multiple of 10. If two consecutive chunks ...

work page

Show all 26 references

[9]

We feed the chunks where NOAS is 2 to the doa2 network with the second input being DOA of the associated event in cartesian xyz coordinates

To determine the remaining information about angles we need to predict the direction of arrival (DOA) of events that start and end while the associated event is happening. We feed the chunks where NOAS is 2 to the doa2 network with the second input being DOA of the associated ...
[10]

Lastly, we predict the events’ classes. If an event has chunks where the event is happening in an isolation (NOAS = 1), then all such chunks are feed to the class network and the most probable class (using soft voting among frames) is taken as a predicted class. If an event ha...

2019
[11]

This dataset constitutes of two parts: the devel- opment and evaluation sets

RESULTS We evaluate our results on TAU Spatial Sound Events 2019 - Ambisonic dataset. This dataset constitutes of two parts: the devel- opment and evaluation sets. The development part consists of 400 recordings with predeﬁned 4-fold cross-validation and the evalua- tion part ...

2019
[12]

Table 5: The comparison of the selected submissions

SUBMISSIONS Overall, we created 4 submissions for the competition: • ConseqFOA (Kapka_SRPOL_task3_2), • ConseqFOA1 (Kapka_SRPOL_task3_3), • ConseqFOAb (Kapka_SRPOL_task3_4), • MLDcT32019 (Lewandowski_SRPOL_task3_1). Table 5: The comparison of the selected submissions. Rank Sub...
[13]

However, we are aware that our solution has some serious limitations and it fails when one wants to consider a more general setup

CONCLUSION We conclude that decomposing the SELD problem into simpler tasks is instinctive and efﬁcient. However, we are aware that our solution has some serious limitations and it fails when one wants to consider a more general setup. For example when there are more than 2 ac...
[14]

Detection and Classiﬁcation of Acoustic Scenes and Events 2019 25–26 October 2019, New York, NY , USA

ACKNOWLEDGEMENT We are most grateful to Zuzanna Kwiatkowska for spending her time on careful reading with a deep understanding the ﬁnal draft of this paper. Detection and Classiﬁcation of Acoustic Scenes and Events 2019 25–26 October 2019, New York, NY , USA

2019
[15]

http://dcase.community/challenge2019/ task-sound-event-localization-and-detection
[16]

Sound event localization and detection of overlapping sources using convolutional recurrent neural networks,

S. Adavanne, A. Politis, J. Nikunen, and T. Virtanen, “Sound event localization and detection of overlapping sources using convolutional recurrent neural networks,”IEEE Journal of Se- lected Topics in Signal Processing, pp. 1–1, 2018

2018
[17]

A multi- room reverberant dataset for sound event localization and uetection,

S. Adavanne, A. Politis, and T. Virtanen, “A multi- room reverberant dataset for sound event localization and uetection,” in Submitted to Detection and Classiﬁcation of Acoustic Scenes and Events 2019 Workshop (DCASE2019) ,

2019
[19]

Adam: A method for stochastic optimization,

D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” 2014. [Online]. Available: https: //arxiv.org/abs/1412.6980

2014 arXiv
[20]

Polyphonic sound event detection and localization using a two-stage strategy,

Y . Cao, Q. Kong, T. Iqbal, F. An, W. Wang, and M. D. Plumbley, “Polyphonic sound event detection and localization using a two-stage strategy,” 2019. [Online]. Available: https://arxiv.org/abs/1905.00268

2019 arXiv
[21]

Two-stage sound event local- ization and detection using intensity vector and generalized cross-correlation,

Y . Cao, T. Iqbal, Q. Kong, M. Galindo, W. Wang, and M. Plumbley, “Two-stage sound event local- ization and detection using intensity vector and generalized cross-correlation,” 2019. [Online]. Avail- able: http://dcase.community/documents/challenge2019/ technical reports/DCASE...

2019
[22]

Data augmenta- tion and prior knowledge-based regularization for sound event localization and detection,

J. Zhang, W. Ding, and L. He, “Data augmenta- tion and prior knowledge-based regularization for sound event localization and detection,” 2019. [Online]. Avail- able: http://dcase.community/documents/challenge2019/ technical reports/DCASE2019 He 97.pdf

2019
[23]

Three-stage approach for sound event lo- calization and detection,

K. Noh, C. Jeong-Hwan, J. Dongyeop, and C. Joon- Hyuk, “Three-stage approach for sound event lo- calization and detection,” 2019. [Online]. Avail- able: http://dcase.community/documents/challenge2019/ technical reports/DCASE2019 Chang 81.pdf

2019
[24]

Squeeze-and-excitation net- works,

J. Hu, L. Shen, and G. Sun, “Squeeze-and-excitation net- works,” inThe IEEE Conference on Computer Vision and Pat- tern Recognition (CVPR), June 2018, pp. 7132–7141

2018
[25]

Specaugment: A simple data augmentation method for automatic speech recognition,

D. S. Park, W. Chan, Y . Zhang, C.-C. Chiu, B. Zoph, E. D. Cubuk, and Q. V . Le, “Specaugment: A simple data augmentation method for automatic speech recognition,”
[26]

Available: https://arxiv.org/abs/1904.08779

[Online]. Available: https://arxiv.org/abs/1904.08779

1904 arXiv
[2019]

Available: https://arxiv.org/abs/1905.08546

[Online]. Available: https://arxiv.org/abs/1905.08546

1905 arXiv

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.

[1] [1]

This year, the task 3 from the IEEE AASP Challenge on Detection and Classiﬁcation of Acoustic Scenes and Events (DCASE 2019) [1] concerned the SELD prob- lem

INTRODUCTION Sound Event Localization and Detection (SELD) is a complex task which naturally appears when one wants to develop a sys- tem that possesses spatial awareness of the surrounding world us- ing multi-channel audio signals. This year, the task 3 from the IEEE AASP Challenge on Detection and Classiﬁcation of Acoustic Scenes and Events (DCASE 2019)...

work page 2019

[2] [2]

Sound source detection, localization and classification using consecutive ensemble of CRNN models

FEATURES The DCASE 2019 task 3 provides two formats of the TAU Spa- tial Sound Events 2019 dataset: ﬁrst order ambisonic ( foa) and 4 channels from a microphone array (mic) [3]. In our method we only use the ambisonic format. arXiv:1908.00766v2 [eess.AS] 5 Sep 2019 Detection and Classiﬁcation of Acoustic Scenes and Events 2019 25–26 October 2019, New York...

work page Pith review arXiv 2019

[3] [3]

Each of these models is a copy of a single SELDnet node with just minor adjustments so that it ﬁts to the speciﬁc subtask and for the regular- ization purpose

ARCHITECTURE As mentioned in the introduction, each of the subtasks ( noas, doa1, doa2 and class) has its own SELDnet-like CRNN. Each of these models is a copy of a single SELDnet node with just minor adjustments so that it ﬁts to the speciﬁc subtask and for the regular- ization purpose. Each of these models takes as an input a ﬁxed length subse- quence o...

work page

[4] [4]

This custom binding of our four models allows us to predict the events’ onset, offset, direction of arrival and class

CONSECUTIVE ENSEMBLE In this section, we introduce and describe the idea of the con- secutive ensemble which is the core of our approach. This custom binding of our four models allows us to predict the events’ onset, offset, direction of arrival and class. 4.1. The algorithm We assume that recordings have at most 2 active sound sources at once and the sou...

work page

[5] [5]

We feed the features to the noas network to predict the num- ber of active sources (NOAS) in each frame

work page

[6] [6]

We transform the predicted NOAS so that each recording starts and ends with no sound sources and the difference of NOAS between each frames is no greater than 1

work page

[7] [7]

If NOAS in two consecutive frames increases, then we predict that a new event happened at the second frame

From the predicted NOAS we deduce the number of events, their onsets and the list of possible offsets for each event. If NOAS in two consecutive frames increases, then we predict that a new event happened at the second frame. If in two consecutive frames NOAS decreases, then we append the ﬁrst frame to all events since last time NOAS was 0 as a possible offset

work page

[8] [8]

We extract chunks (intervals of equal NOAS) of audio where the predicted NOAS equals 1 and we feed it to doa1 network

In order to determine which offset corresponds to which event we use the doa1 network. We extract chunks (intervals of equal NOAS) of audio where the predicted NOAS equals 1 and we feed it to doa1 network. For each chunk where NOAS was 1 we predict the average azimuth and elevation, and we round it to the closest multiple of 10. If two consecutive chunks ...

work page