{"id":"0fdd73f4-3024-4fd6-b6fd-c1e1f2ed5102","arxiv_id":"2507.09570","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"Replacing the Conformer decoder in pretrained PSELDnet with a bidirectional Mamba plus asymmetric convolution reports 39.6% versus 38.2% stereo SELD F20 on the DCASE2025 development set, using 76M versus 210M parameters.","lead":"This paper builds a stereo sound event localization and detection system that replaces the costly Transformer decoder in a pretrained audio model with a cheaper bidirectional Mamba sequence model. On the DCASE2025 development set it reports a slightly better localization F1 score with far fewer parameters, though the numbers have no error bars and the claimed code link is not usable as printed.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The headline gain is supported by one checkpoint per model on the development set: the 39.6% vs 38.2% F20 advantage over the Conformer is within plausible seed-to-seed variation, and the official-evaluation F20 (31.0%) shows a large unexplained generalization gap.","rationale":"The Reader's weakest assumption and my load-bearing concern coincide: the paper's central quantitative comparison is a set of unreplicated point estimates on the development set, with no variance or significance assessment. The reader's conditional verdict already captures this by requiring replication and artifact repair. My stress-test adds two concrete observations that strengthen the concern rather than change the verdict: (1) the reported F20 gap between BiMambaAC and the Conformer is only 1.4 points, which is small relative to typical seed-to-seed variation in SELD training; and (2) the same system drops from 39.6% F20 on the development set to 31.0% on the official evaluation set, a large unexplained gap that casts doubt on the stability of the reported advantage. The paper's architecture description is plausible and the efficiency improvement is consistent with the parameter/MAC tables, but the accuracy claim needs repeated-seed evidence before it can be accepted as more than a favorable run. The proposed test—multi-seed reruns with paired confidence intervals—would settle whether the +3.4 point F20 advantage over BiMamba and the +1.4 point advantage over the Conformer are reproducible or within noise. If the advantages survive, the conditional acceptance can be lifted; if they do not, the claim should be downgraded. No further concern rises to the level of changing the reader's disposition, so the verdict should remain unchanged at CONDITIONAL until the test is performed.","tokens_in":7752,"tokens_out":4443,"duration_ms":52121,"concrete_test":"Rerun the fine-tuning scenario (Table II and Table III) with at least 10 random seeds per configuration, using the same data splits, optimizer schedule, and validation-F1 checkpoint selection. Report mean ± std for F20, DOAE, and RDE, and compute paired 95% confidence intervals or a paired test for BiMambaAC vs Conformer and BiMambaAC vs BiMamba. If the F20 advantage over Conformer does not exceed the CI width (or if the CI includes 0), the headline claim is not established. As a secondary check, obtain the official-evaluation predictions and verify whether the 31.0% F20 is consistent with the development-set performance under the same model.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim (Table II) rests on a single-run comparison. Section III-C states only that the checkpoint with the highest validation location-dependent F1-score is selected; no seeds, repeated runs, confidence intervals, or significance tests are reported. On a 13-class SELD task, 1–2 point F20 differences between architectures are commonly within run-to-run variance, and the paper's own numbers reveal fragility: the same BiMambaAC system achieves 39.6% F20 on the development set (Table II) but 31.0% F20 on the official challenge evaluation (Table IV), an 8.6-point drop that the text neither explains nor analyzes. The other key comparisons (BiMambaAC vs BiMamba, +3.4 points; ablation without asymmetric convolution, -6.4 points) are likewise unaccompanied by variance estimates. If the 3.4-point gap is real and reproducible, the architecture story holds; if it is a favorable checkpoint draw, the efficiency claims survive but the accuracy advantage does not. Because the paper's conclusion depends on these point estimates, the load-bearing assumption is that a single development-set run is representative of expected performance.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a stereo sound event localization and detection (SELD) system for DCASE2025 Task 3, built by replacing the Conformer decoder of the pre-trained PSELDnet with a bidirectional Mamba (BiMamba) module augmented by asymmetric convolutions (BiMambaAC). Stereo input is converted to a pseudo-FOA representation via a mid/side-style transformation. The authors evaluate the model on the DCASE2025 development set and the official challenge evaluation, reporting location-dependent F1 (F20), DOA error, relative distance error, parameter count, and MACs. In the fine-tuning scenario, BiMambaAC reaches 39.6% F20 with 76M parameters, outperforming the Conformer-based PSELDnet (38.2% F20, 210M parameters) and BiMamba (36.2% F20, 178M parameters). An ablation study shows drops of 8.6 points without bidirectional processing and 6.4 points without asymmetric convolution. The official evaluation result for the proposed system is 31.0% F20, ranking 9th in the challenge.","tokens_in":8040,"tokens_out":3311,"duration_ms":35981,"significance":"If the reported gains are reproducible, the paper makes a useful contribution: it demonstrates that a Mamba-based decoder can replace a Conformer decoder in a pre-trained SELD system with a favorable accuracy-efficiency trade-off, while the public code release supports reproducibility. The ablation study is internally consistent, and the comparison includes several relevant baselines. However, the central quantitative claim rests on single-run development-set scores without variance estimates, and the large unexplained gap between the development-set F20 (39.6%) and the official evaluation F20 (31.0%) weakens confidence in the headline improvement. The significance is therefore conditional on additional evidence of stability and on a credible explanation of the generalization gap.","major_comments":[{"comment":"The central claim that BiMambaAC outperforms BiMamba by +3.4% F20 and the Conformer baseline by +1.4% F20 is supported only by a single development-set run per model, with checkpoint selection based on validation F1 and no repeated seeds, confidence intervals, or significance tests. This is a load-bearing issue because 1–2 point F20 differences are plausibly within run-to-run variance for a 13-class SELD task. Moreover, the same system achieves 39.6% F20 on the development set (Table II) but only 31.0% F20 on the official evaluation (Table IV), an 8.6-point drop that the text neither explains nor analyzes. The authors should report multiple runs with variance, and either analyze the development-to-evaluation gap or temper the accuracy claims accordingly.","section":"Section IV-B, Tables II and IV"},{"comment":"The abstract states that the proposed method performs better than both the baseline and the original PSELDnet with a Conformer decoder, but this is not true in the from-scratch scenario: Table I shows Conformer at 32.7% F20 versus BiMambaAC at 32.1% F20. The improvement over Conformer only holds in the fine-tuning scenario (Table II). The claim should be explicitly qualified to the fine-tuning setting, and the from-scratch result should be discussed rather than ignored.","section":"Section IV-A, Table I vs. Abstract"},{"comment":"The ablation result without bidirectional Mamba is 31.0% F20, which is numerically identical to the official-evaluation F20 of the proposed system in Table IV. This coincidence is not addressed. It is important to clarify which configuration was actually submitted to the challenge: if the official submission was the full BiMambaAC model, the equality is coincidental and should be noted; if the official submission differed, the paper should state so explicitly. Otherwise, the reader cannot assess whether the challenge result reflects the proposed architecture.","section":"Section IV-B, Table III and Table IV"}],"minor_comments":[{"comment":"The output equation is written as y(t) = C^T h'(t) + Dx(t), but the standard SSM output is y(t) = C^T h(t) + Dx(t). As written, the output depends on the time derivative of the state, which is inconsistent with the surrounding derivation.","section":"Section II-A, Eq. (2)"},{"comment":"The expression for \\bar{B} uses (\\Delta A)^{-1}, which is not well-defined when \\Delta A is singular. The zero-order-hold discretization is usually given in integral or series form; the authors should provide the nonsingular-free formulation.","section":"Section II-A, Eq. (3)"},{"comment":"The hyperparameters are not fixed across models: batch size ranges from 32 to 256 and weight decay from 1e-4 to 5e-6 depending on model size. This makes the architecture comparisons less controlled; the authors should state whether the same tuning procedure was applied to all baselines or acknowledge the potential confound.","section":"Section III-C"},{"comment":"The heading contains a typo: 'Asymtric' should be 'Asymmetric'.","section":"Table III"},{"comment":"The GitHub URL contains spaces ('https://github.com/ alexandergwm/DCASE2025 TASK3 Stereo PSELD Mamba'); the link should be corrected and verified.","section":"Abstract and Section V"},{"comment":"Several references in the related work are by the same author group ([4]–[7], [9], [21], [22]); including them is fine, but the authors may wish to cite additional independent works on Mamba-based SELD to broaden the context.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The paper is a straightforward empirical challenge-style contribution. The main risk is not circularity but statistical fragility: the headline accuracy gain is within plausible seed noise, and the official evaluation number is far below the development-set number. The authors need to either add repeated runs/variance or explicitly reposition the paper as an efficiency-oriented contribution where the accuracy claim is secondary. Also, the self-citation density in the references is noticeable; I would not block on it, but the editor may wish to keep it in mind during the review process."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the paper does something sensible and reports it honestly. The new bits are the specific combination of pretrained PSELDnet with a BiMamba decoder that uses asymmetric convolutions, and the pseudo-FOA stereo adaptation. The components themselves are not new—BiMamba already replaces Conformer in SELD-Mamba—but the combination and the measurements on the DCASE2025 stereo task are not in the cited literature.\n\nWhat it does well: the ablation is internally consistent, the parameter and MACs reporting is useful, and the comparison set is relevant. The paper does not oversell; its conclusions match its tables. The pseudo-FOA conversion is stated as an engineering assumption, not hidden.\n\nThe soft spots are mostly about evidence quality. The central comparison—39.6% vs 38.2% F20 over Conformer in Table II—comes from one checkpoint per model, chosen by validation F1, with no seeds, repeated runs, or confidence intervals. On a 13-class SELD task, a 1.4-point difference is within plausible run-to-run variation. That concern becomes sharper when you look at Table IV: the same BiMambaAC system gets 31.0% F20 on the official evaluation set, an 8.6-point drop that the text neither explains nor analyzes. That gap does not disprove the architecture story, but it does mean the dev-set gain might be partly selection noise.\n\nThe ablation drops are also from single runs. Removing asymmetric convolution costs 6.4 points; removing bidirectional Mamba costs 8.6 points. Those are large effects but unquantified. A serious revision should add repeated runs with error bars and some analysis of the dev-to-eval gap.\n\nMinor issues: the GitHub URL as printed has spaces and is not directly usable, and the pseudo-FOA assumption (X and Z set to zero) is unvalidated against alternatives. The self-citations are not load-bearing and are relevant background, so I do not see a citation problem.\n\nWho this is for: anyone working on efficient SELD architectures, especially in the DCASE challenge space. It is a modest, useful data point, not a breakthrough.\n\nRecommendation: it deserves a serious referee. I would send it to peer review, but ask for repeated runs, variance estimates, and a discussion of the generalization gap before acceptance.","headline":"Useful efficiency-oriented tweak on PSELDnet for stereo SELD, but the headline accuracy gain rests on single-run dev-set scores with no variance estimates.","tokens_in":8541,"tokens_out":2313,"would_cite":false,"duration_ms":24115,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Replacing the Conformer decoder of a pretrained PSELDnet with a bidirectional Mamba block and asymmetric convolutions yields the best stereo sound event localization and detection scores on the DCASE2025 development set while using 76M…","keywords":["stereo sound event localization and detection","BiMamba","state space models","PSELDnet pretraining","asymmetric convolution","DCASE2025 Task 3","efficient audio models","Multi-ACCDOA"],"falsifier":"Run BiMambaAC and the Conformer baseline 10 times each with different seeds on the DCASE2025 Task 3 development set, selecting checkpoints by validation F1 as the paper does, and compare the mean and standard deviation of F20; the claimed advantage is unsupported if the Conformer mean reaches or exceeds the BiMambaAC mean within one standard deviation.","tokens_in":7564,"feed_emoji":"🎧","tokens_out":7640,"duration_ms":70380,"temperature":0.7,"pith_summary":"The paper claims that a stereo sound-event localization and detection system can be made both more accurate and far cheaper by swapping the heavy Conformer decoder of a pretrained PSELDnet for a bidirectional Mamba block augmented with asymmetric convolutions. On the DCASE2025 Task 3 development set, the resulting BiMambaAC model reaches 39.6% location-dependent F1 at a 20-degree threshold and 15.8 degrees DOA error using 76M parameters, beating the Conformer baseline (38.2% F20, 210M parameters) and the plain BiMamba variant (36.2% F20, 178M parameters). The paper also reports that the same model placed 9th in the DCASE 2025 Task 3 challenge with a single model, no external data, and only simple channel-swapping augmentation. The practical stakes are efficiency: state-space sequence models offer a path to competitive SELD without Transformer-scale compute.","feed_headline":"BiMamba beats Conformer for stereo SELD with 76M parameters","feed_subtitle":"A state-space decoder plus asymmetric convolution beats the Transformer baseline at one-third the parameters.","key_machinery":"The load-bearing component is the BiMamba2DAC block: a bidirectional state-space sequence layer (two Mamba directions over time) combined with an asymmetric convolution pathway that applies 1D convolutions separately over the time axis and the frequency axis instead of a single 2D convolution. This block replaces the Conformer decoder in the pretrained PSELDnet pipeline, while a temporal module restores and aligns the temporal resolution after CNN14 pooling. The argument is carried by two mechanisms: state-space selectivity lets the model track overlapping events without quadratic attention cost, and time-frequency decoupling extracts complementary features at lower compute.","core_discovery":"The paper claims that, in the fine-tuning scenario, its proposed BiMambaAC model — a pretrained CNN14 encoder with a bidirectional Mamba decoder augmented by asymmetric convolution operating separately on time and frequency — is the best overall system on the DCASE2025 Task 3 stereo SELD development set. It reports 39.6% location-dependent F1 at a 20° threshold (F20) and 15.8° DOA error with 76M parameters, compared with 38.2% F20 for the original Conformer model at 210M parameters, 36.2% F20 for plain BiMamba at 178M parameters, and 35.1% F20 for HTS-AT at 28M parameters. The ablation attributes an 8.6-point F20 drop to removing bidirectional processing and a 6.4-point drop to removing asymmetric convolution, and the paper notes that increasing the state dimension from 64 to 128 adds only 0.3% F20 while costing 6GB of extra GPU memory. On the official challenge evaluation dataset the same single-model system scores 31.0% F20, which the paper reports as 9th place with no external data and only simple channel-swapping augmentation.","pith_inferences":["An untested extension of the same recipe would attach the BiMamba decoder to a different pretrained encoder, such as HTS-AT; the paper's pretraining results suggest that encoder also benefits strongly from pretraining, which would separate encoder gains from decoder gains.","Because the stereo-to-pseudo-FOA conversion zeroes the X and Z components, the model is only estimating left-right direction; learning to predict the missing components from stereo cues, or adding microphone-array geometry, is a natural way to extend the system toward full 3D localization.","The asymmetric-convolution block is a generic time-frequency decoupling module, so the same swap could be tested in single-channel sound event detection, where the compute savings would be even more visible."],"forward_implications":["Fine-tuning from pretrained PSELDnet weights helps every architecture in the comparison, and BiMambaAC is the top performer on the development set (39.6% F20, 15.8° DOAE).","Both new components matter: taking away bidirectional processing costs 8.6 points of F20, and taking away asymmetric convolution costs 6.4 points.","Distilling the Conformer's heavy decoder into a 76M-parameter BiMamba decoder preserves or improves accuracy while reducing parameters by more than half and MACs by 39% relative to the plain BiMamba variant.","The efficiency gain is operational: the default dstate=64 configuration is preferred over dstate=128 because the larger state adds only 0.3% F20 but 6GB of training-time GPU memory.","A single-model system with no external training data and only channel swapping can reach 31.0% F20 on the official evaluation set, ahead of some ensemble submissions that rely on external data."],"supporting_citations":[{"why":"Supplies the pretrained CNN14 audio encoder that both the Conformer baseline and the proposed BiMambaAC decoder are attached to.","marker":"[15]"},{"why":"Supplies the PSELDnet pretrained weights and the fine-tuning protocol from which all Table II models start.","marker":"[14]"},{"why":"Introduces the Mamba selective state-space sequence model that BiMamba makes bidirectional.","marker":"[18]"},{"why":"Provides the Mamba2/SSD module used to compute intra-chunk and inter-chunk state evolution in the BiMamba blocks.","marker":"[19]"},{"why":"Prior SELD work replacing the Conformer with BiMamba in EINV2; the comparison that motivates the decoder swap.","marker":"[24]"},{"why":"Defines the Multi-ACCDOA output format, the prediction target and evaluation format used throughout.","marker":"[12]"},{"why":"The DCASE2025 Task 3 stereo SELD development and evaluation dataset on which all tables report scores.","marker":"[29]"}],"fun_headline_variants":["BiMambaAC tops Conformer on stereo SELD: 39.6% vs 38.2% F20","76M-parameter BiMamba outdoes 210M Conformer in stereo SELD","BiMamba state-space decoder beats Transformer for stereo SELD at 76M params","Asymmetric conv and BiMamba lift stereo SELD to 39.6% F20, 76M params","Stereo SELD with BiMamba: +1.4 F20 over Conformer, 76M params"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The comparison assumes that a single development-set run per model, with the checkpoint chosen by validation F1 and no significance testing, is stable enough to support the 1.4-point F20 advantage over the Conformer baseline.","fun_headline_variants_meta":{"raw":{"variants":["BiMambaAC tops Conformer on stereo SELD: 39.6% vs 38.2% F20","76M-parameter BiMamba outdoes 210M Conformer in stereo SELD","BiMamba state-space decoder beats Transformer for stereo SELD at 76M params","Asymmetric conv and BiMamba lift stereo SELD to 39.6% F20, 76M params","Stereo SELD with BiMamba: +1.4 F20 over Conformer, 76M params"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001265,"raw_usage":{"total_tokens":5190,"prompt_tokens":971,"completion_tokens":4219,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":587,"completion_tokens_details":{"reasoning_tokens":4086}},"tokens_in":587,"tokens_out":4219,"duration_ms":32808,"temperature":1.0,"reasoning_tokens":4086,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T17:52:41.779252+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run BiMambaAC and the Conformer baseline 10 times each with different seeds on the DCASE2025 Task 3 development set, selecting checkpoints by validation F1 as the paper does, and compare the mean and standard deviation of F20; the claimed advantage is unsupported if the Conformer mean reaches or exceeds the BiMambaAC mean within one standard deviation.","supporting_citations":[{"cited_title":"PSELDNets: Pre-trained Neural Networks on Large-scale Synthetic Datasets for Sound Event Localization and Detection,","cited_arxiv_id":null,"evidence_quote":"Supplies the PSELDnet pretrained weights and the fine-tuning protocol from which all Table II models start."},{"cited_title":"Mamba: Linear-Time Sequence Modeling with Selective State Spaces,","cited_arxiv_id":null,"evidence_quote":"Introduces the Mamba selective state-space sequence model that BiMamba makes bidirectional."},{"cited_title":"Multi-ACCDOA: Localizing and Detecting Overlapping Sounds from the Same Class with Auxiliary Duplicating Permutation Invariant Training","cited_arxiv_id":"2110.07124","evidence_quote":"Defines the Multi-ACCDOA output format, the prediction target and evaluation format used throughout."},{"cited_title":"DCASE2025 Task3 Stereo SELD Dataset,","cited_arxiv_id":null,"evidence_quote":"The DCASE2025 Task 3 stereo SELD development and evaluation dataset on which all tables report scores."}],"review_version":1}