{"id":"a7a8dcf9-d873-47f1-b15b-1fb3e3a13d33","arxiv_id":"2508.15632","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"ASCMamba, a dual-path Mamba model with location and time conditioning, wins the APSIPA ASC 2025 challenge with 64.4% macro accuracy.","lead":"A challenge entry combines sound spectrogram features with location and time text embeddings in a Mamba-based network, plus a pseudo-labeling scheme. The audio-only version of the system placed first in the APSIPA ASC 2025 acoustic scene classification challenge, beating the official baseline by 6.2%.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Test-set-aware validation split (Sec. IV-A) makes the reported 64.4% blind-test result an overestimate of generalization; the audio-only variant was selected using evaluation-set feature statistics.","rationale":"The reader identified the same load-bearing concern: using the official evaluation set to construct validation splits and select the final model. This is a real methodological weakness that inflates the apparent robustness of the audio-only variant and makes the official test score less interpretable as a generalization result. The concern does not invalidate the fact that the system officially ranked first—that is an externally verified outcome—but it does weaken the paper's scientific claims about robustness and about the merits of multimodal fusion. The reader's conditional verdict is appropriate: the core empirical result is credible though over-interpreted, and the methodology has a transparency gap that should be fixed. I agree with the reader's assessment, so no verdict change is needed. The concrete test I propose directly addresses the weak point by checking whether the selection decision is robust to removing evaluation-set influence.","tokens_in":7209,"tokens_out":2878,"duration_ms":34637,"concrete_test":"Inspect the released repository (github.com/S-Orion/ASCMamba) to verify whether the validation-split creation in Section IV-A consumes any features from the official evaluation set. If it does, re-run the model-selection step using only development-set-based splits (e.g., random split or leave-one-city-out cross-validation) and retrain/submit the preferred variant; if the audio-only variant no longer wins on the official test set, then the reported 6.2% advantage is an artifact of test-set-aware selection. If the code does not actually use evaluation-set features, or if the audio-only preference persists under evaluation-set-free selection, the concern is mitigated.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central empirical claim—first place with 64.4% macro ACC and a 6.2% gain over baseline—is officially evaluated, but the paper's own model-selection procedure compromises the interpretability of that result. In Section IV-A, the authors extract LMFB features from the official evaluation set, compute cosine similarity with validation-set features, and use a 0.9 threshold to partition the validation set into Valid-Easy and Valid-Hard. They then use performance on Valid-Hard (including shuffled location/time and unseen-location simulations, Figs. 3–4) to conclude that the audio-only variant is more robust and to submit that variant. This is test-set-aware model selection: the validation splits are engineered to match the evaluation set's feature distribution, so the choice between ASCMamba w/ L&RT and w/o L&RT is informed by the unlabeled test distribution. As a result, the reported 64.4% accuracy is not an unbiased estimate of performance on a new sample from the same task; it reflects selection using statistics of the particular official test set. The paper's broader claim that audio-only is preferable under distribution shift is therefore not established as a general property—it may be an artifact of overfitting to the evaluation set's specific feature statistics. A secondary issue is that the winning model discards location/time metadata, which contradicts the paper's multimodal framing; the core improvement over the baseline appears to come from audio-only modeling with the dual-path Mamba and pseudo-labeling, not from multimodal fusion.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper describes ASCMamba, the proposed system for the APSIPA ASC 2025 Grand Challenge acoustic scene classification task. The architecture combines a DenseEncoder with dual-path Mamba blocks that model time and frequency dependencies, and optionally conditions the feature representation on location/record-time text embeddings through conditional layer normalization. A two-step pseudo-labeling scheme is used to exploit unlabeled development data. The authors report that the submitted system, labeled 'ASCMamba w/o L&RT' (i.e., without location and record-time metadata), achieved rank 1 on the official blind test set with 64.4% macro accuracy, a 6.2% absolute improvement over the SE-Trans baseline.","tokens_in":7612,"tokens_out":4312,"duration_ms":51613,"significance":"The headline result is supported by an external, organizer-scored blind evaluation, which is a strong form of validation. The paper also releases code, model, and pre-trained checkpoints. If the architectural claims were fully substantiated, the work would provide a useful data point for Mamba-based state-space models in acoustic scene classification. However, the scientific contribution is weakened by three issues: (1) the validation protocol uses the official evaluation set's feature statistics to construct validation splits and select the final model variant; (2) the central robustness comparison is reported without error bars or significance tests; and (3) the submitted model is audio-only, while the paper's framing emphasizes a multimodal contribution. These issues do not invalidate the official rank, but they limit the strength of the generalizations drawn from the experiments.","major_comments":[{"comment":"The validation-set creation protocol uses the official evaluation set to engineer Valid-Easy/Valid-Hard splits: LMFB features are extracted from the evaluation set, cosine similarity with validation features is computed, and a threshold of 0.9 assigns samples to Valid-Hard. The subsequent choice of ASCMamba w/o L&RT as the submitted system is therefore informed by the unlabeled test distribution. The official 64.4% score is still a legitimate blind-label evaluation, but it is not an unbiased estimate of performance on a new sample from the same task, and the conclusion that audio-only modeling is more robust under distribution shift is not established as a general property. Please report a leave-one-location-out or city-separated validation, or explicitly frame the result as transductive model selection.","section":"IV-A, V"},{"comment":"All reported comparisons between ASCMamba w/ L&RT and w/o L&RT are single-run accuracy values with no error bars, confidence intervals, or significance tests. The key difference on Valid-Hard (96.41% vs 94.14%) is small and could be within run-to-run variation. Without multiple seeds or a paired test across the same validation folds, the robustness claim is not statistically supported. This is load-bearing because the decision to submit the audio-only variant rests on these figures.","section":"Table III, Figs. 3-4"},{"comment":"The paper is framed as a 'multimodal' system, and the abstract states that ASCMamba 'integrates audio and textual information.' However, the submitted system is explicitly 'ASCMamba w/o L&RT', meaning the conditional location/time pathway is disabled. The official rank-1 result is thus for the audio-only model, while the multimodal variant is only an ablation experiment. This mismatch should be clarified in the abstract, the architecture description, and the conclusion; as written, a reader could reasonably infer that the winning system uses the proposed multimodal fusion.","section":"Abstract, III-A, IV-A, V"},{"comment":"The two-step pseudo-labeling mechanism is presented as a key contribution, but no ablation isolates its effect. Table III compares only the baseline, w/ L&RT, and w/o L&RT; no experiment compares training with and without pseudo-labeling, or with a simpler one-step confidence threshold. Without such an ablation, the claimed benefit of the two-step intersection strategy is not demonstrated.","section":"III-C, V"}],"minor_comments":[{"comment":"The statement that the evaluation set has 12 locations with 6 overlapping in the development set is useful, but the source of this information should be cited (challenge description or official dataset documentation).","section":"II-B"},{"comment":"Equation (2) is hard to parse: the notation 'c,i⊃c' and the phrase 'since i ⊃ c' are confusing. Typically one would write c ∈ {1,...,10}, i ∈ {1,2}, with the constraint that class c belongs to the indoor/outdoor group i. Please rewrite the equation and the accompanying explanation.","section":"III-B, Eq. (2)"},{"comment":"The pretraining stage on TAU UAS 2020 and CochlScene is described, but the scene mapping/merging procedure in Table I is not detailed. Since this is a nonstandard pretraining corpus, the mapping rules should be specified or the merged dataset released.","section":"IV-C"},{"comment":"The text says 'Experimental results in demonstrate' — a missing noun. Also, the conclusion repeats 'demonstrate' twice. Please proofread.","section":"V"},{"comment":"Reference [13] is formatted as 'K. D. B. J. Adam et al.'; the correct citation is Kingma and Ba, 'Adam: A Method for Stochastic Optimization.'","section":"References"},{"comment":"The two figures have identical axes and similar curves; the difference between shuffling spatiotemporal metadata (Fig. 3) and replacing locations with unseen ones (Fig. 4) should be stated clearly in each caption.","section":"Fig. 3 and Fig. 4"}],"recommendation":"major_revision","confidential_remarks":"The use of the official evaluation set to construct validation splits is the most significant concern. If the challenge rules prohibited any use of unlabeled test data for model selection, this would become a much more serious issue; even if permitted, the paper should clearly state the transductive nature of the selection procedure. The mismatch between the 'multimodal' framing and the audio-only submitted system also needs to be addressed before publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a solid competition write-up, not a scientific breakthrough. The headline result—first place on APSIPA ASC 2025 with 64.4% macro ACC, 6.2% over baseline—is externally scored and credible. The architecture (DenseEncoder + dual-path Mamba with conditional layer normalization) is a reasonable integration of known pieces, and the two-step pseudo-labeling scheme is sensible. Credit where due: they report the official blind test ranking, release code and checkpoints, and are transparent about their validation procedure.\n\nThe soft spots, in order of size. First, Section IV-A is the real concern: they use the official evaluation set's feature statistics to split their validation into Valid-Easy/Hard, then use those splits to choose the audio-only variant for submission. That makes the 64.4% score an honest number but a selected number—the choice between w/ and w/o L&RT was informed by the unlabeled test distribution. So the paper's broader conclusion that audio-only is more robust under domain shift is not established as a general property; it may be specific to this test set. This doesn't sink the result, but it should be stated plainly.\n\nSecond, the multimodal framing is overstated. The submitted system is the audio-only version, so 'ASCMamba' as the winning system is not actually the multimodal model. The multimodal variant is only presented as an ablation. A more accurate title and framing would help.\n\nThird, the ablation results have no error bars or significance tests. For a challenge report that's forgivable, but I wouldn't read too much into small gaps like 94.14 vs 96.41.\n\nThe data/citation pattern is fine; the key references are there, and the external baselines are appropriate.\n\nFor whom: anyone working on ASC challenges, Mamba for audio, or semi-supervised audio classification will get some value. It's a legitimate technical report, and the first-place result deserves citation. I'd accept it for peer review at a workshop or venue where challenge systems are in scope, but I'd ask the authors to fix the framing and add a limitation note about the test-set-aware model selection.","headline":"First-place challenge system, credible score, but the audio-only variant was chosen using evaluation-set statistics, so the robustness claim is weaker than it looks.","tokens_in":8041,"tokens_out":2113,"would_cite":true,"duration_ms":23591,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper's central claim is that a Mamba-based dual-path network, ASCMamba, wins the APSIPA ASC 2025 acoustic scene classification challenge at 64.4% macro accuracy, and that its audio-only variant is the right submission because location","keywords":["acoustic scene classification","Mamba state space model","multimodal fusion","conditional layer normalization","semi-supervised pseudo-labeling","domain shift","APSIPA ASC 2025 Grand Challenge"],"falsifier":"Retrain and select the model with strict city-separated cross-validation — holding out the six cities that appear only in the evaluation set, and never extracting any features from the evaluation set itself — then compare the audio-only and multimodal variants on those held-out cities. If the audio-only variant no longer beats the multimodal variant, the reported robustness advantage of dropping location and time is an artifact of tuning on the official test set's statistics rather than a property of the model.","tokens_in":7144,"feed_emoji":"🏆","tokens_out":6445,"duration_ms":68040,"temperature":0.7,"pith_summary":"This paper reports the system that won the APSIPA ASC 2025 acoustic scene classification challenge: ASCMamba, a neural network built on Mamba state-space blocks that scan spectrograms along both time and frequency axes. The challenge added two extra inputs beyond audio — the recording location and the recording time — and the paper's design folds these into the network through a conditioned normalization layer. The surprising result is that the winning submission is the audio-only version: under domain shift (unseen cities or shuffled metadata), the multimodal version degrades, while audio-only stays accurate. The paper argues that context information should be used only when it is reliable, and contributes a two-step pseudo-labeling scheme to exploit the large unlabeled portion of the challenge data.","feed_headline":"Mamba model wins scene-classification challenge at 64.4%","feed_subtitle":"Location and time context boosts accuracy on familiar data but hurts when the test city is new.","key_machinery":"The Dual-path Mamba block carries the argument: the spectrogram feature is reshaped into two sequences — each frequency bin as a sequence over record time, and each frame as a sequence over frequency — and each is run through a Mamba selective state-space block, giving long-range modeling along both axes. Multimodal fusion is carried by Conditional Layer Normalization, CLN(x,c) = γ(c)·LN(x) + β(c), where the affine parameters are generated from the projected text embedding, so location and recording time modulate the feature space. A two-step pseudo-labeling scheme extends the semi-supervised pipeline: the top 90% of unlabeled clips by confidence receive labels from the fine-tuned ASCMamba;","core_discovery":"On its own terms, the paper establishes two things. First, the ASCMamba architecture — a DenseEncoder front-end feeding dual Mamba paths, one scanning along time and one along frequency — achieves 64.4% macro accuracy on the challenge's official blind test set, beating the SE-Trans baseline by 6.2% and every other team. Second, its ablations show that the multimodal variant (fusing location and recording-time text embeddings via Conditional Layer Normalization) is stronger when distributions match (97.71% vs 93.75% on the easy validation split) but weaker when the test environment shifts (94.14% vs 96.41% on the hard split, degrading further as metadata is shuffled or locations become unseen","pith_inferences":["The validation protocol extracts feature statistics from the official evaluation set to build the Valid-Hard splits; a strictly city-separated cross-validation that never touches the evaluation set would test whether the audio-only advantage and the 6.2% margin are genuine or partly an artifact of tuning on the test set's statistics.","The pattern — large gains from conditioning on in-distribution metadata, clear losses under shift — suggests the multimodal branch learns a location/time shortcut rather than a generative context; a testable extension would train with deliberately corrupted or dropped metadata to force the network not to rely on it.","The same Conditional Layer Normalization mechanism could carry other auxiliary text such as device, weather, or annotator notes, and the paper's trade-off implies such conditioning should include a confidence gate on the metadata itself.","The paper does not separate the winning margin into contributions from the Mamba backbone, the pseudo-labeling, and the decision to drop location and time; swapping one component at a time in the four-stage pipeline would isolate which piece drives the 6.2% improvement."],"forward_implications":["If correct, the architecture shows that Mamba-based state-space modeling over both time and frequency axes is a strong backbone for environmental audio, competitive with attention-based backbones on this task.","The ablation result implies a practical rule for any multimodal audio system: fuse location or time metadata only when the deployment distribution matches the training distribution; under domain shift, audio-only models are the safer choice.","The two-step pseudo-labeling intersection scheme expands usable training data from 1,740 official labeled clips to a much larger set, evidence that semi-supervised training recovers accuracy from unlabeled audio when official labels are scarce.","The paper's comparison with the runner-up systems attributes their lower scores to insufficient handling of positional-information reliance, which positions robustness to metadata shift as the decisive factor separating the top teams."],"supporting_citations":[{"why":"Supplies the Mamba selective state-space block that forms the backbone of the dual-path sequence modeling.","marker":"[5]"},{"why":"TAU Urban Acoustic Scenes 2020 Mobile is one of the two pre-training corpora whose scene labels are merged.","marker":"[7]"},{"why":"CochlScene is the crowdsourced pre-training corpus that scales up the labeled data for initial training.","marker":"[8]"},{"why":"SE-Trans is the official challenge baseline that must be beaten and the base architecture for the improved second pseudo-labeling system.","marker":"[9]"},{"why":"Supplies the dense-connectivity design that the DenseEncoder front-end is built on for hierarchical spectral features.","marker":"[10]"},{"why":"Provides the score-fusion rule used to combine the ten-class and indoor/outdoor classifier outputs in the improved SE-Trans.","marker":"[11]"},{"why":"Distinguishes the previous ICME 2024 ASC challenge from this one, framing the new multimodal and semi-supervised task setting.","marker":"[4]"}],"fun_headline_variants":["Mamba model wins acoustic scene challenge at 64.4%","Mamba wins scene task, but new cities break the fusion","ASCMamba: time-frequency Mamba beats baseline by 6.2%","Location-aware Mamba wins acoustic scene challenge, fails on unseen cities"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The model-selection evidence assumes the authors may extract feature statistics from the official evaluation set to build their validation splits, so the conclusion that the audio-only model is more robust holds only if that peek at the test set's statistics did not steer the choice.","fun_headline_variants_meta":{"raw":{"variants":["Mamba model wins acoustic scene challenge at 64.4%","Mamba wins scene task, but new cities break the fusion","ASCMamba: time-frequency Mamba beats baseline by 6.2%","Location-aware Mamba wins acoustic scene challenge, fails on unseen cities"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000727,"raw_usage":{"total_tokens":3103,"prompt_tokens":764,"completion_tokens":2339,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":508,"completion_tokens_details":{"reasoning_tokens":2262}},"tokens_in":508,"tokens_out":2339,"duration_ms":17403,"temperature":1.0,"reasoning_tokens":2262,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T17:45:43.681603+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Retrain and select the model with strict city-separated cross-validation — holding out the six cities that appear only in the evaluation set, and never extracting any features from the evaluation set itself — then compare the audio-only and multimodal variants on those held-out cities. If the audio-only variant no longer beats the multimodal variant, the reported robustness advantage of dropping location and time is an artifact of tuning on the official test set's statistics rather than a property of the model.","supporting_citations":[{"cited_title":"Tau urban acoustic scenes 2020 mobile development dataset [data set],","cited_arxiv_id":null,"evidence_quote":"TAU Urban Acoustic Scenes 2020 Mobile is one of the two pre-training corpora whose scene labels are merged."},{"cited_title":"Cochlscene: Acquisition of acoustic scene data using crowdsourcing,","cited_arxiv_id":null,"evidence_quote":"CochlScene is the crowdsourced pre-training corpus that scales up the labeled data for initial training."},{"cited_title":"A squeeze-and-excitation and transformer-based cross- task model for environmental sound recognition,","cited_arxiv_id":null,"evidence_quote":"SE-Trans is the official challenge baseline that must be beaten and the base architecture for the improved second pseudo-labeling system."},{"cited_title":"Densenets reloaded: Paradigm shift beyond resnets and vits,","cited_arxiv_id":null,"evidence_quote":"Supplies the dense-connectivity design that the DenseEncoder front-end is built on for hierarchical spectral features."},{"cited_title":"A two-stage approach to device-robust acoustic scene classification,","cited_arxiv_id":null,"evidence_quote":"Provides the score-fusion rule used to combine the ten-class and indoor/outdoor classifier outputs in the improved SE-Trans."}],"review_version":1}