{"id":"20fea0c9-6fc4-48a9-8416-3a91411a2ec1","arxiv_id":"2507.12042","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"low","formal_verification":"none","parameter_count":3,"one_line_summary":"It introduces a stereo-audio sound event localization benchmark with onscreen/offscreen classification, and finds the audiovisual baseline's onscreen judgments are near chance.","lead":"This paper introduces a new benchmark for locating and identifying sound events in ordinary stereo audio and video, along with a new sub-task of deciding whether each sound source is visible onscreen. The benchmark, built from existing spatial recordings, comes with baseline models that still struggle with distance and onscreen/offscreen judgments.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Onscreen/offscreen labels may be computed from front-back-folded azimuths, so physically offscreen sources can be labeled onscreen; this would invalidate the audiovisual sub-task's ground truth.","rationale":"The paper's central claim is that the released dataset and baseline demonstrate that stereo SELD is feasible, including the new audiovisual sub-task of onscreen/offscreen classification. For that claim to hold, the ground-truth labels must be internally consistent with the audio and video content. The reader's weakest assumption already flags the front-back folding step as a potential source of label corruption, and I agree with that general concern. However, I see a more specific and more damaging version: the text of Section 2.2 can be read as computing the onscreen/offscreen label after folding, which would systematically mislabel a large fraction of back-hemisphere events. If this reading is correct, the audiovisual baseline metrics are not meaningful, and the paper's conclusion in Section 8 that 'current models underutilize visual information' would need to be reinterpreted as a consequence of untrainable labels. The paper's own Section 7 already notes that the onscreen/offscreen accuracy is near the data-distribution prior, so label quality is central to interpreting the results. A one-day code inspection and relabeling check would settle this. I therefore recommend keeping the reader's CONDITIONAL verdict, with the condition extended to include verification of the label-generation order. If the check passes, the concern is resolved and the existing rationale for conditional acceptance stands unchanged.","tokens_in":9367,"tokens_out":5831,"duration_ms":73159,"concrete_test":"Run the released data generator on at least 1,000 randomly sampled clips and, for every annotated event, compare (a) the released onscreen/offscreen label with (b) FOV membership computed from the unfolded rotated azimuth (i.e., |azimuth_from_camera_center| <= 50° before folding). Count events with (b)=false but (a)=true, stratified by original azimuth hemisphere. If back-hemisphere events with folded angles inside the FOV are labeled onscreen, the audiovisual ground truth is corrupted and the sub-task needs relabeling or a corrected label-generation description. If the count is zero, the current labels are sound and the existing conditional-acceptance rationale stands.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"Section 2.2 describes a label pipeline with a potential internal inconsistency. It states that rotated back-hemisphere azimuth labels are folded to the front, and then 'we compare the new DOA labels with the azimuthal range of the FOV'. If 'new' means post-fold, then a physically offscreen source can become onscreen: with a 100° FOV centered at 0°, a source at 170° azimuth (behind, not visible in the perspective video) folds to 10° and is labeled onscreen. Because the M/S stereo conversion in Equations (1)-(2) encodes only the W and Y (left-right) components and discards the X (front-back) component, the stereo signal for 170° and 10° is indistinguishable; a model trained on such labels receives contradictory audiovisual targets, since the video frame contains no source for an event labeled onscreen. This would explain the baseline's onscreen/offscreen accuracy of roughly 80%, which Section 7 itself notes is close to the 77.5% always-offscreen prior. The released data generator makes the intended order checkable; if FOV membership is computed from the unfolded rotated azimuth before folding, the concern is refuted. This is an internal correctness risk, distinct from the external-validity question of whether M/S conversion represents real stereo media.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper introduces the DCASE2025 Task 3 stereo sound event localization and detection (SELD) challenge, including a new audiovisual sub-task of onscreen/offscreen classification. The authors construct the DCASE2025 Task3 Stereo SELD Dataset by converting STARSS23 FOA recordings to M/S stereo audio and 360-degree video to perspective video with a fixed 100-degree horizontal FOV. They describe a baseline CRNN audio-only model and an audiovisual model with a ResNet-50 visual encoder and transformer fusion, along with modified evaluation metrics: azimuth-only DOAE, relative distance error, onscreen/offscreen accuracy, and a localization-dependent F-score that incorporates an on/off-screen correctness requirement for the audiovisual track. Baseline results are reported on development and evaluation splits, and a dataset-bias analysis shows that the onscreen/offscreen accuracy of the audiovisual baseline is close to the 77.5% always-offscreen prior and that distance estimation underperforms a class-average bias baseline.","tokens_in":9621,"tokens_out":3035,"duration_ms":35258,"significance":"The paper is a challenge description and baseline paper, not a methods paper, and it offers a substantial community resource: a publicly released dataset, a reproducible data-generation pipeline, baseline code, and a transparent bias analysis. The M/S stereo conversion equations (1)-(2) are simple and correct given the stated ACN/SN3D convention, and the evaluation protocol is clearly specified. If the ground-truth construction is sound, the dataset enables a new research direction of stereo SELD for regular media content. The paper also honestly reports that the audiovisual baseline struggles to exploit visual information, which is a useful finding for the community. However, the central claim that stereo SELD is feasible and shows promise rests on single-run baseline scores whose statistical reliability is not established, and one aspect of the label-generation pipeline creates a potential correctness risk for the audiovisual ground truth.","major_comments":[{"comment":"The order of operations for computing the onscreen/offscreen labels is critically ambiguous. The text states that rotated azimuth labels pointing to the back hemisphere are folded front-back, and then 'we compare the new DOA labels with the azimuthal range of the FOV'. If 'new' means post-fold, then a physically offscreen source behind the camera (e.g., azimuth 170 degrees) folds to 10 degrees and is labeled onscreen even though it is not visible in the perspective video. Because the M/S conversion in Eqs. (1)-(2) discards the X (front-back) component, the stereo audio for a source at 170 degrees is identical to that at 10 degrees; a model trained on such labels receives contradictory audiovisual targets. The reported audiovisual onscreen/offscreen accuracy of 80.0% (Table 1) and 77.8% (Table 2) is close to the 77.5% always-offscreen prior, so the ambiguity is not resolved by the experimental results. Please state explicitly whether FOV membership is computed from the rotated azimuth before or after front-back folding, and if necessary release a corrected version of the dataset labels.","section":"Section 2.2"},{"comment":"All results in Tables 1 and 2 are reported as single-run scores without error bars, confidence intervals, or significance tests. Several comparisons are numerically close: the audiovisual model outperforms the audio-only model by 4.0 points in macro F20/1 on the development set but by only 1.4 points on the evaluation set, and the onscreen/offscreen accuracy is within a few points of the 77.5% majority-class prior as the paper itself notes in Section 7. In the absence of repeated-run variability or a paired test, the conclusion in Section 8 that 'stereo SELD is feasible, with initial results indicating promise' is not strongly supported. I recommend adding multiple training runs or at least a clear statement that the reported numbers are single-run and should be interpreted cautiously.","section":"Section 6"},{"comment":"The dataset-bias analysis is a strength, but the statement that the onscreen/offscreen accuracy 'is around a random guess adjusted to the data distribution' is under-specified. A classifier that always predicts the majority class would achieve approximately 77.5% accuracy, and the audiovisual baseline achieves 80.0% / 77.8%. To better assess whether the model learns any onscreen/offscreen information, the paper should report balanced accuracy, ROC-AUC, or per-class accuracies in addition to the raw accuracy; otherwise the difference of 2.5 points or 0.3 points is difficult to interpret as evidence of learning. This is especially important because the onscreen/offscreen correctness is part of the primary ranking metric F20/1/onoff in the audiovisual track.","section":"Section 7"}],"minor_comments":[{"comment":"There is a typo in 'Apart form last-year metrics' which should be 'Apart from last-year metrics'.","section":"Section 5"},{"comment":"The phrase 'in for the audiovisual track' is ungrammatical; it should likely be 'and for the audiovisual track' or 'for the audiovisual track'.","section":"Section 5"},{"comment":"The sentence 'The details of the sampling procedure are provided below' is redundant because the sampling details immediately follow; consider removing it or restructuring the paragraph.","section":"Section 2.2"},{"comment":"The example captions describe left/right positions of speakers, but the figure would benefit from marking the onscreen/offscreen labels and the FOV boundary so readers can visually verify the conversion described in Section 2.2.","section":"Figure 1"},{"comment":"The description of the multi-ACCDOA output as Cartesian [x,y] coordinates is a bit confusing given that the paper earlier states DOA estimation is azimuth-only; please clarify that the y-component here represents the front-back dimension and is intentionally underdetermined by stereo audio, or explain how the baseline handles this ambiguity during training.","section":"Section 3"}],"recommendation":"major_revision","confidential_remarks":"The paper is a useful challenge-resource paper with a valuable dataset release. The onscreen/offscreen ground-truth ambiguity in Section 2.2 is the most serious issue because it directly affects the validity of the audiovisual sub-task and the ranking metric; this needs to be resolved before the challenge results can be considered reliable. The lack of error bars is a standard limitation for challenge baselines but should be acknowledged explicitly. As a challenge description paper, the fit to the journal/conference venue is appropriate, and the authors' honest bias analysis should be commended."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a solid challenge setup paper that does what it says—new stereo SELD dataset, new on/offscreen sub-task, baseline, metrics, and released code. Worth a careful read. But I'm worried the onscreen/offscreen ground truth may be computed from the front-back-folded azimuths, which would corrupt the audiovisual sub-task's labels.\n\nWhat's genuinely new: the DCASE2025 Task3 Stereo SELD Dataset derived from STARSS23, the mid-side conversion from FOA, the perspective video simulation with a 100-degree FOV, the on/offscreen accuracy metric, and the audio-only plus audiovisual baseline. The paper is admirably honest about dataset bias: Section 7 explicitly notes that 77.5% of sources are offscreen and that the AV baseline's on/off accuracy is close to that prior, and that distance estimation is worse than simply using the class-conditional average distance. Transparency like this is exactly what you want in a challenge description.\n\nThe dicey part is Section 2.2. The text says rotated back-hemisphere azimuth labels are folded to the front, and then 'we compare the new DOA labels with the azimuthal range of the FOV.' If 'new' means post-fold, a source physically at 170 degrees azimuth, behind the camera, gets folded to 10 degrees and is labeled onscreen even though the perspective video cannot show it. The stereo signal from M/S conversion is left-right symmetric, so 170 and 10 degrees produce identical audio; the AV model would see contradictory targets. That would explain the near-prior on/off accuracy. The data generator code is public, so the intended order is checkable, but the natural reading of the text points to the wrong order. This is an internal-validity risk for Track B, distinct from the external question of whether M/S represents real stereo media.\n\nOther concerns are minor: baseline numbers are single-run without error bars; the architecture is a light adaptation of last year's; and the conclusion's 'promise for practical applications' overreaches given F-scores around 22-27%, though that is boilerplate for these papers.\n\nWho this is for: anyone working on SELD, audiovisual fusion, or benchmark design. It deserves a serious referee; the dataset and task framing are useful to the field. But before Track B results can be trusted, the authors should confirm the code path and clarify whether on/off labels are computed from the folded or unfolded azimuth. If post-fold, the on/off sub-task's ground truth is wrong and the AV baseline's ~80% accuracy is meaningless. If pre-fold, the concern dissolves. As written, I would ask for a code check and a rerun or clear statement before accepting.","headline":"A genuinely useful stereo SELD challenge setup, but the on/off ground truth may hinge on ordering of azimuth folding versus FOV check—verify the code before trusting Track B.","tokens_in":10211,"tokens_out":3320,"would_cite":false,"duration_ms":36044,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Stereo audio alone can support sound event localization and detection in the left-right plane, according to the new DCASE2025 Task 3 dataset and baselines.","keywords":["stereo sound event localization and detection","DCASE2025 Task 3","mid-side stereo conversion","onscreen/offscreen classification","distance estimation","multi-ACCDOA","audiovisual fusion","STARSS23"],"falsifier":"Count, across the development set, the frames in which two simultaneously active sources have back-hemisphere azimuths that fold onto the same front angle; if such collisions are frequent, the folded labels are systematically ambiguous. Alternatively, run the baseline on genuine coincident-pair or binaural stereo recordings with known source positions and compare the localization F-score with the FOA-derived test set; a large drop would show the synthetic stereo conversion does not stand in for real stereo media.","tokens_in":9194,"feed_emoji":"🎧","tokens_out":5748,"duration_ms":63033,"temperature":0.7,"pith_summary":"This paper establishes a new challenge task: detecting and localizing sound events from ordinary stereo audio, the format of most media content. It argues that even though stereo has inherent front-back and top-bottom ambiguities, a system can still estimate an event's left-right azimuth and its distance, and it introduces a dataset of 40,000 five-second clips converted from real spatial recordings to support that claim. The paper also adds a sub-task for audiovisual input: deciding whether each sound source is onscreen or offscreen, which matters when the video sees only a 100-degree field of view. The baseline results show stereo SELD is feasible, with the audiovisual model exceeding the audio-only model on detection and angular accuracy but not on distance, and an onscreen/offscreen accuracy of 80 percent that largely mirrors the dataset's 1:3 onscreen ratio.","feed_headline":"Stereo audio can locate sound events, new dataset shows","feed_subtitle":"A 41.7-hour benchmark converts four-channel recordings to everyday stereo and finds left-right localization works.","key_machinery":"The load-bearing conversion is the mid-side stereo extraction $L(n) = W(n) + Y(n)$, $R(n) = W(n) - Y(n)$, which turns the zeroth-order omnidirectional signal and left-right dipole of ACN/SN3D FOA into a coincident stereo pair. Around this sit three other mechanism pieces: the folding of back-hemisphere azimuths to the front to remove the front-back ambiguity, the multi-ACCDOA output format that couples class activity with Cartesian DOA and distance regressors so overlapping same-class events can be tracked, and the localization-dependent F-score with a 20-degree and relative-distance-1 threshold that ranks systems. For the audiovisual track, a pre-trained ResNet-50 feature stream is fused with audio through transformer cross-attention, and a binary neuron predicts onscreen/offscreen status. Together these define the task's input, output, and success measure.","core_discovery":"The central claim is that stereo-only audio carries enough spatial information to support sound event localization and detection, provided the task is reformulated around the left-right axis. The dataset is built by taking STARSS23 first-order Ambisonics and perspective-video recordings, rotating them to a random viewing direction, and converting the four-channel audio to mid-side stereo with $L = W + Y$ and $R = W - Y$; azimuth labels in the back hemisphere are folded to the front because stereo cannot distinguish front from back, and elevation labels are discarded. The baseline uses a convolutional recurrent network with multi-head attention and a multi-ACCDOA output that predicts up to three simultaneous events per class with Cartesian $[x, y]$ DOA coordinates and a distance value. Evaluated on held-out clips, the audio-only baseline reaches a macro localization-dependent F-score of 26.1 percent and the audiovisual baseline 27.5 percent, with onscreen/offscreen accuracy near 78 percent. The paper's stated conclusion is that stereo SELD is feasible and the dataset supports reproducible training and evaluation.","pith_inferences":["The paper does not test on stereo recorded natively; a natural extension is to record or collect true mid-side or binaural material and see whether the FOA-derived stereo baseline transfers.","Because the viewing angle is chosen randomly and 77.5 percent of sources are offscreen, an onscreen/offscreen model can score near 80 percent by always guessing 'offscreen'; future splits should balance onscreen events to make the metric informative.","Folding back azimuths means a sound at +120 degrees and one at +60 degrees both map to the same front label; an ambiguity-aware training loss or a front/back confidence output might extract more spatial information than the current framing.","The distance bias analysis suggests that class-conditioned distance priors are a strong baseline for this dataset, so future systems should be compared against such priors to demonstrate genuine distance learning."],"forward_implications":["SELD can be evaluated on ordinary media audio rather than specialized four-channel formats, widening the task to consumer recordings and video content.","A system that only sees the left-right axis can still localize events well enough to beat chance, with the audio-only baseline reaching a macro F20°/1 of 26.1 percent on the evaluation set.","The audiovisual track's ranking metric now requires a correct onscreen/offscreen call for a detection to count, so visual context is scored as part of localization.","Distance estimation remains the weakest link; replacing the model's distance with the class-average distance of the estimated class improves or matches the baselines, indicating the models have not learned true distance cues."],"supporting_citations":[{"why":"Supplies the real-scene spatial recordings and annotation pipeline that the new dataset builds on.","marker":"[15]"},{"why":"Provides the audio-visual spatial recordings from which the stereo clips and perspective videos are sampled and converted.","marker":"[16]"},{"why":"Defines the previous challenge baseline architecture and evaluation that this year's audio-only model is adapted from.","marker":"[17]"},{"why":"Introduces the distance-estimation formulation and evaluation that the stereo task adopts alongside azimuth DOA.","marker":"[25]"},{"why":"Provides the two-channel versus four-channel comparison that justifies the mid-side stereo conversion used in the dataset.","marker":"[29]"},{"why":"Defines the multi-ACCDOA output format the baseline uses to handle overlapping same-class events.","marker":"[32]"},{"why":"The synthetic spatial audio simulation library used to generate extra FOA training data with aligned annotations.","marker":"[33]"}],"fun_headline_variants":["Stereo audio locates sound events in new dataset","Left-right sound localization from stereo audio","New benchmark turns stereo into SELD","Stereo SELD: onscreen/offscreen included","DCASE2025 stereo audio DOA benchmark"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole benchmark rests on the assumption that stereo signals produced by the mid-side equations from four-channel recordings behave like stereo from real media, and that folding rear sounds forward does not create contradictory labels.","fun_headline_variants_meta":{"raw":{"variants":["Stereo audio locates sound events in new dataset","Left-right sound localization from stereo audio","New benchmark turns stereo into SELD","Stereo SELD: onscreen/offscreen included","DCASE2025 stereo audio DOA benchmark"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000225,"raw_usage":{"total_tokens":1519,"prompt_tokens":1054,"completion_tokens":465,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":670,"completion_tokens_details":{"reasoning_tokens":395}},"tokens_in":670,"tokens_out":465,"duration_ms":5303,"temperature":1.0,"reasoning_tokens":395,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T16:54:56.401088+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Count, across the development set, the frames in which two simultaneously active sources have back-hemisphere azimuths that fold onto the same front angle; if such collisions are frequent, the folded labels are systematically ambiguous. Alternatively, run the baseline on genuine coincident-pair or binaural stereo recordings with known source positions and compare the localization F-score with the FOA-derived test set; a large drop would show the synthetic stereo conversion does not stand in for real stereo media.","supporting_citations":[{"cited_title":"Starss22: A dataset of spatial recordings of real scenes with spatiotem- poral annotations of sound events,","cited_arxiv_id":null,"evidence_quote":"Supplies the real-scene spatial recordings and annotation pipeline that the new dataset builds on."},{"cited_title":"Starss23: An audio-visual dataset of spatial recordings of real scenes with spatiotemporal annotations of sound events,","cited_arxiv_id":null,"evidence_quote":"Provides the audio-visual spatial recordings from which the stereo clips and perspective videos are sampled and converted."},{"cited_title":"Baseline models and evaluation of sound event localization and detection with distance estimation in dcase 2024 challenge,","cited_arxiv_id":null,"evidence_quote":"Defines the previous challenge baseline architecture and evaluation that this year's audio-only model is adapted from."},{"cited_title":"Sound event detection and localization with distance estimation,","cited_arxiv_id":null,"evidence_quote":"Introduces the distance-estimation formulation and evaluation that the stereo task adopts alongside azimuth DOA."},{"cited_title":"Two vs. four-channel sound event localization and detection,","cited_arxiv_id":null,"evidence_quote":"Provides the two-channel versus four-channel comparison that justifies the mid-side stereo conversion used in the dataset."},{"cited_title":"Multi-accdoa: Localizing and detecting overlapping sounds from the same class with auxiliary duplicating permutation invariant training,","cited_arxiv_id":null,"evidence_quote":"Defines the multi-ACCDOA output format the baseline uses to handle overlapping same-class events."},{"cited_title":"Spatial scaper: a library to simulate and augment soundscapes for sound event localization and detection in realistic rooms,","cited_arxiv_id":null,"evidence_quote":"The synthetic spatial audio simulation library used to generate extra FOA training data with aligned annotations."}],"review_version":1}