ASCMamba: Multimodal Time-Frequency Mamba for Acoustic Scene Classification

Bochao Sun; Dong Wang; Han Yin; Jun Yang; ZhanLong Yang

REVIEW 4 major objections 6 minor 16 references

ASCMamba: Multimodal Time-Frequency Mamba for Acoustic Scene Classification

T0 review · 4 major / 6 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper's central claim is that a Mamba-based dual-path network, ASCMamba, wins the APSIPA ASC 2025 acoustic scene classification challenge at 64.4% macro accuracy, and that its audio-only variant is the right submission because location

desk verdict First-place challenge system, credible score, but the audio-only variant was chosen using evaluation-set statistics, so the robustness claim is weaker than it looks. read the letter →

arxiv 2508.15632 v2 pith:OKSEDDHB submitted 2025-08-21 cs.SD

Bochao Sun , Dong Wang , ZhanLong Yang , Jun Yang , Han Yin This is my paper

classification cs.SD

keywords acousticsceneclassificationMambastatespacemodelmultimodalfusionconditionallayernormalizationsemi-supervisedpseudo-labelingdomainshiftAPSIPAASC2025GrandChallenge

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper reports the system that won the APSIPA ASC 2025 acoustic scene classification challenge: ASCMamba, a neural network built on Mamba state-space blocks that scan spectrograms along both time and frequency axes. The challenge added two extra inputs beyond audio — the recording location and the recording time — and the paper's design folds these into the network through a conditioned normalization layer. The surprising result is that the winning submission is the audio-only version: under domain shift (unseen cities or shuffled metadata), the multimodal version degrades, while audio-only stays accurate. The paper argues that context information should be used only when it is reliable, and contributes a two-step pseudo-labeling scheme to exploit the large unlabeled portion of the challenge data.

What carries the argument

The Dual-path Mamba block carries the argument: the spectrogram feature is reshaped into two sequences — each frequency bin as a sequence over record time, and each frame as a sequence over frequency — and each is run through a Mamba selective state-space block, giving long-range modeling along both axes. Multimodal fusion is carried by Conditional Layer Normalization, CLN(x,c) = γ(c)·LN(x) + β(c), where the affine parameters are generated from the projected text embedding, so location and recording time modulate the feature space. A two-step pseudo-labeling scheme extends the semi-supervised pipeline: the top 90% of unlabeled clips by confidence receive labels from the fine-tuned ASCMamba;

What would settle it

Retrain and select the model with strict city-separated cross-validation — holding out the six cities that appear only in the evaluation set, and never extracting any features from the evaluation set itself — then compare the audio-only and multimodal variants on those held-out cities. If the audio-only variant no longer beats the multimodal variant, the reported robustness advantage of dropping location and time is an artifact of tuning on the official test set's statistics rather than a property of the model.

Watch

Extended reading notes

Core claim

On its own terms, the paper establishes two things. First, the ASCMamba architecture — a DenseEncoder front-end feeding dual Mamba paths, one scanning along time and one along frequency — achieves 64.4% macro accuracy on the challenge's official blind test set, beating the SE-Trans baseline by 6.2% and every other team. Second, its ablations show that the multimodal variant (fusing location and recording-time text embeddings via Conditional Layer Normalization) is stronger when distributions match (97.71% vs 93.75% on the easy validation split) but weaker when the test environment shifts (94.14% vs 96.41% on the hard split, degrading further as metadata is shuffled or locations become unseen

Load-bearing premise

The model-selection evidence assumes the authors may extract feature statistics from the official evaluation set to build their validation splits, so the conclusion that the audio-only model is more robust holds only if that peek at the test set's statistics did not steer the choice.

Editorial extensions

If this is right

If correct, the architecture shows that Mamba-based state-space modeling over both time and frequency axes is a strong backbone for environmental audio, competitive with attention-based backbones on this task.
The ablation result implies a practical rule for any multimodal audio system: fuse location or time metadata only when the deployment distribution matches the training distribution; under domain shift, audio-only models are the safer choice.
The two-step pseudo-labeling intersection scheme expands usable training data from 1,740 official labeled clips to a much larger set, evidence that semi-supervised training recovers accuracy from unlabeled audio when official labels are scarce.
The paper's comparison with the runner-up systems attributes their lower scores to insufficient handling of positional-information reliance, which positions robustness to metadata shift as the decisive factor separating the top teams.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The validation protocol extracts feature statistics from the official evaluation set to build the Valid-Hard splits; a strictly city-separated cross-validation that never touches the evaluation set would test whether the audio-only advantage and the 6.2% margin are genuine or partly an artifact of tuning on the test set's statistics.
The pattern — large gains from conditioning on in-distribution metadata, clear losses under shift — suggests the multimodal branch learns a location/time shortcut rather than a generative context; a testable extension would train with deliberately corrupted or dropped metadata to force the network not to rely on it.
The same Conditional Layer Normalization mechanism could carry other auxiliary text such as device, weather, or annotator notes, and the paper's trade-off implies such conditioning should include a confidence gate on the metadata itself.
The paper does not separate the winning margin into contributions from the Mamba backbone, the pseudo-labeling, and the decision to drop location and time; swapping one component at a time in the four-stage pipeline would isolate which piece drives the 6.2% improvement.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Quick take: this is a solid competition write-up, not a scientific breakthrough. The headline result—first place on APSIPA ASC 2025 with 64.4% macro ACC, 6.2% over baseline—is externally scored and credible. The architecture (DenseEncoder + dual-path Mamba with conditional layer normalization) is a reasonable integration of known pieces, and the two-step pseudo-labeling scheme is sensible. Credit where due: they report the official blind test ranking, release code and checkpoints, and are transparent about their validation procedure.

The soft spots, in order of size. First, Section IV-A is the real concern: they use the official evaluation set's feature statistics to split their validation into Valid-Easy/Hard, then use those splits to choose the audio-only variant for submission. That makes the 64.4% score an honest number but a selected number—the choice between w/ and w/o L&RT was informed by the unlabeled test distribution. So the paper's broader conclusion that audio-only is more robust under domain shift is not established as a general property; it may be specific to this test set. This doesn't sink the result, but it should be stated plainly.

Second, the multimodal framing is overstated. The submitted system is the audio-only version, so 'ASCMamba' as the winning system is not actually the multimodal model. The multimodal variant is only presented as an ablation. A more accurate title and framing would help.

Third, the ablation results have no error bars or significance tests. For a challenge report that's forgivable, but I wouldn't read too much into small gaps like 94.14 vs 96.41.

The data/citation pattern is fine; the key references are there, and the external baselines are appropriate.

For whom: anyone working on ASC challenges, Mamba for audio, or semi-supervised audio classification will get some value. It's a legitimate technical report, and the first-place result deserves citation. I'd accept it for peer review at a workshop or venue where challenge systems are in scope, but I'd ask the authors to fix the framing and add a limitation note about the test-set-aware model selection.

Referee Report

4 major / 6 minor

Summary. The paper describes ASCMamba, the proposed system for the APSIPA ASC 2025 Grand Challenge acoustic scene classification task. The architecture combines a DenseEncoder with dual-path Mamba blocks that model time and frequency dependencies, and optionally conditions the feature representation on location/record-time text embeddings through conditional layer normalization. A two-step pseudo-labeling scheme is used to exploit unlabeled development data. The authors report that the submitted system, labeled 'ASCMamba w/o L&RT' (i.e., without location and record-time metadata), achieved rank 1 on the official blind test set with 64.4% macro accuracy, a 6.2% absolute improvement over the SE-Trans baseline.

Significance. The headline result is supported by an external, organizer-scored blind evaluation, which is a strong form of validation. The paper also releases code, model, and pre-trained checkpoints. If the architectural claims were fully substantiated, the work would provide a useful data point for Mamba-based state-space models in acoustic scene classification. However, the scientific contribution is weakened by three issues: (1) the validation protocol uses the official evaluation set's feature statistics to construct validation splits and select the final model variant; (2) the central robustness comparison is reported without error bars or significance tests; and (3) the submitted model is audio-only, while the paper's framing emphasizes a multimodal contribution. These issues do not invalidate the official rank, but they limit the strength of the generalizations drawn from the experiments.

major comments (4)

[IV-A, V] The validation-set creation protocol uses the official evaluation set to engineer Valid-Easy/Valid-Hard splits: LMFB features are extracted from the evaluation set, cosine similarity with validation features is computed, and a threshold of 0.9 assigns samples to Valid-Hard. The subsequent choice of ASCMamba w/o L&RT as the submitted system is therefore informed by the unlabeled test distribution. The official 64.4% score is still a legitimate blind-label evaluation, but it is not an unbiased estimate of performance on a new sample from the same task, and the conclusion that audio-only modeling is more robust under distribution shift is not established as a general property. Please report a leave-one-location-out or city-separated validation, or explicitly frame the result as transductive model selection.
[Table III, Figs. 3-4] All reported comparisons between ASCMamba w/ L&RT and w/o L&RT are single-run accuracy values with no error bars, confidence intervals, or significance tests. The key difference on Valid-Hard (96.41% vs 94.14%) is small and could be within run-to-run variation. Without multiple seeds or a paired test across the same validation folds, the robustness claim is not statistically supported. This is load-bearing because the decision to submit the audio-only variant rests on these figures.
[Abstract, III-A, IV-A, V] The paper is framed as a 'multimodal' system, and the abstract states that ASCMamba 'integrates audio and textual information.' However, the submitted system is explicitly 'ASCMamba w/o L&RT', meaning the conditional location/time pathway is disabled. The official rank-1 result is thus for the audio-only model, while the multimodal variant is only an ablation experiment. This mismatch should be clarified in the abstract, the architecture description, and the conclusion; as written, a reader could reasonably infer that the winning system uses the proposed multimodal fusion.
[III-C, V] The two-step pseudo-labeling mechanism is presented as a key contribution, but no ablation isolates its effect. Table III compares only the baseline, w/ L&RT, and w/o L&RT; no experiment compares training with and without pseudo-labeling, or with a simpler one-step confidence threshold. Without such an ablation, the claimed benefit of the two-step intersection strategy is not demonstrated.

minor comments (6)

[II-B] The statement that the evaluation set has 12 locations with 6 overlapping in the development set is useful, but the source of this information should be cited (challenge description or official dataset documentation).
[III-B, Eq. (2)] Equation (2) is hard to parse: the notation 'c,i⊃c' and the phrase 'since i ⊃ c' are confusing. Typically one would write c ∈ {1,...,10}, i ∈ {1,2}, with the constraint that class c belongs to the indoor/outdoor group i. Please rewrite the equation and the accompanying explanation.
[IV-C] The pretraining stage on TAU UAS 2020 and CochlScene is described, but the scene mapping/merging procedure in Table I is not detailed. Since this is a nonstandard pretraining corpus, the mapping rules should be specified or the merged dataset released.
[V] The text says 'Experimental results in demonstrate' — a missing noun. Also, the conclusion repeats 'demonstrate' twice. Please proofread.
[References] Reference [13] is formatted as 'K. D. B. J. Adam et al.'; the correct citation is Kingma and Ba, 'Adam: A Method for Stochastic Optimization.'
[Fig. 3 and Fig. 4] The two figures have identical axes and similar curves; the difference between shuffling spatiotemporal metadata (Fig. 3) and replacing locations with unseen ones (Fig. 4) should be stated clearly in each caption.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation; the headline result is anchored by an external blind-test evaluation.

full rationale

The paper's central claim—64.4% macro accuracy, first place, 6.2% over the baseline—is based on the official blind test set scored by the challenge organizers. This is external evidence, not a quantity derived from the model's own outputs or fitted parameters. The pseudo-labeling pipeline uses the model's own predictions to construct additional training data, but this affects training only; the final evaluation is independent. The only notable methodological issue is in Section IV-A, where LMFB features from the official evaluation set are used to construct Valid-Easy/Valid-Hard and to select the audio-only variant. That is test-set-aware model selection and a legitimate generalization concern, but it is not circularity in the derivation sense: no equation or fitted parameter is renamed as a prediction, and the official score was still measured on the hidden test. Self-citations to SE-Trans, the ICME challenge, and other teams' reports are contextual and not load-bearing; no uniqueness theorem is imported from the authors' prior work, and no ansatz is smuggled in via citation. The paper presents an empirical system, not a derivation chain that reduces to its inputs.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The paper is purely empirical. It relies on hand-chosen thresholds and standard domain assumptions about feature representations and metadata relevance. No new physical or conceptual entities are introduced; location and time embeddings are standard text representations.

free parameters (4)

Cosine similarity threshold (0.9) for Valid-Easy/Valid-Hard split = 0.9
Hand-chosen in Section IV-A; determines the composition of the validation splits used for model selection.
Pseudo-label confidence cutoff (top 90%) = 90%
Hand-chosen in Section III-C; determines which unlabeled samples are used directly for fine-tuning.
Learning rate = 0.0001
Set in Section IV-C; standard choice, not tuned systematically.
Dropout rate = 0.1
Set in Section IV-C; standard regularization hyperparameter.

assumptions (4)

domain assumption Log-mel spectrogram features (64 filter banks, 500 frames) are a sufficient input representation.
Used throughout; no comparison to other front-ends like raw audio or larger filter banks.
domain assumption Location and record time metadata are directly related to scene labels.
Section II-B argues correlation from the development set; this motivates the multimodal conditioning.
ad hoc to paper The official evaluation set's feature statistics are a valid proxy for the hidden test distribution, allowing construction of Valid-Hard.
Section IV-A uses cosine similarity between evaluation and validation features; this assumption drives model selection and is not independently validated.
domain assumption Pseudo-labels agreed upon by two models are reliable enough to fine-tune the final model.
Section III-C assumes intersection of predictions reduces noise; no analysis of pseudo-label accuracy is provided.

how reviews work

0 comments

Cite this review

Pith. "Pith review of ASCMamba: Multimodal Time-Frequency Mamba for Acoustic Scene Classification." pith.science (2026). https://pith.science/paper/OKSEDDHB

@misc{pith2026250815632,
  author       = {Pith},
  title        = {Pith review of: ASCMamba: Multimodal Time-Frequency Mamba for Acoustic Scene Classification},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OKSEDDHB}},
  note         = {Machine review of arXiv:2508.15632}
}

read the original abstract

Acoustic Scene Classification (ASC) is a fundamental problem in computational audition, which seeks to classify environments based on the distinctive acoustic features. In the ASC task of the APSIPA ASC 2025 Grand Challenge, the organizers introduce a multimodal ASC task. Unlike traditional ASC systems that rely solely on audio inputs, this challenge provides additional textual information as inputs, including the location where the audio is recorded and the time of recording. In this paper, we present our proposed system for the ASC task in the APSIPA ASC 2025 Grand Challenge. Specifically, we propose a multimodal network, ASCMamba, which integrates audio and textual information for fine-grained acoustic scene understanding and effective multimodal ASC. The proposed ASCMamba employs a DenseEncoder to extract hierarchical spectral features from spectrograms, followed by a dual-path Mamba blocks that capture long-range temporal and frequency dependencies using Mamba-based state space models. In addition, we present a two-step pseudo-labeling mechanism to generate more reliable pseudo-labels. Results show that the proposed system outperforms all the participating teams and achieves a 6.2% improvement over the baseline. Code, model and pre-trained checkpoints are available at https://github.com/S-Orion/ASCMamba.git.

Figures

Figures reproduced from arXiv: 2508.15632 by the authors.

**Figure 2.** Overview of the proposed ASCMamba, which is composed of a DenseEncoder and a Dual-path Mamba Block. [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗

**Figure 3.** The performance of the two systems after shuffling the Valid-Hard [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗

Figures from the paper (1 more)

**Figure 4.** Figure 4: The performance of the two systems with the gradual increase in the [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

16 extracted references · 13 canonical work pages

[1]

Acoustic scene classification: Classifying environments from the sounds they produce,

D. Barchiesi, D. Giannoulis, D. Stowell, and M. D. Plumbley, “Acoustic scene classification: Classifying environments from the sounds they produce,” IEEE Signal Processing Magazine , vol. 32, no. 3, pp. 16–34, 2015

work page 2015
[2]

Exploring text-queried sound event detection with audio source separation,

H. Yin, J. Bai, Y . Xiao, et al. , “Exploring text-queried sound event detection with audio source separation,” in IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , IEEE, 2025, pp. 1–5

work page 2025
[3]

Multi-granularity acous- tic information fusion for sound event detection,

H. Yin, J. Chen, J. Bai, et al., “Multi-granularity acous- tic information fusion for sound event detection,” Signal Processing, vol. 227, p. 109 691, 2025

work page 2025
[4]

Description on ieee icme 2024 grand challenge: Semi-supervised acoustic scene classification under domain shift,

J. Bai, M. Wang, H. Liu, et al. , “Description on ieee icme 2024 grand challenge: Semi-supervised acoustic scene classification under domain shift,” arXiv preprint arXiv:2402.02694, 2024

arXiv 2024
[5]

Mamba: Linear-time sequence modeling with selective state spaces,

A. Gu and T. Dao, “Mamba: Linear-time sequence modeling with selective state spaces,” arXiv preprint arXiv:2312.00752, 2023

arXiv 2023
[6]

Attention is all you need,

A. Vaswani, N. Shazeer, N. Parmar, et al., “Attention is all you need,” Advances in neural information process- ing systems, vol. 30, 2017

work page 2017
[7]

Tau urban acoustic scenes 2020 mobile development dataset [data set],

H. Toni, M. Annamaria, and V . Tuomas, “Tau urban acoustic scenes 2020 mobile development dataset [data set],” Zenodo, 2020

work page 2020
[8]

Cochlscene: Acquisition of acoustic scene data using crowdsourcing,

I.-Y . Jeong and J. Park, “Cochlscene: Acquisition of acoustic scene data using crowdsourcing,” in Asia- Pacific Signal and Information Processing Association Annual Summit and Conference, IEEE, 2022, pp. 17–21

work page 2022

Show all 16 references

[9]

A squeeze-and-excitation and transformer-based cross- task model for environmental sound recognition,

J. Bai, J. Chen, M. Wang, M. S. Ayub, and Q. Yan, “A squeeze-and-excitation and transformer-based cross- task model for environmental sound recognition,” IEEE Transactions on Cognitive and Developmental Systems , vol. 15, no. 3, pp. 1501–1513, 2023

2023
[10]

Densenets reloaded: Paradigm shift beyond resnets and vits,

D. Kim, B. Heo, and D. Han, “Densenets reloaded: Paradigm shift beyond resnets and vits,” in Euro- pean Conference on Computer Vision , Springer, 2024, pp. 395–415

2024
[11]

A two-stage approach to device-robust acoustic scene classification,

H. Hu, C.-H. H. Yang, X. Xia, et al. , “A two-stage approach to device-robust acoustic scene classification,” in IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE, 2021, pp. 845– 849

2021
[12]

Librosa: Audio and music signal analysis in python.,

B. McFee, C. Raffel, D. Liang, et al., “Librosa: Audio and music signal analysis in python.,” SciPy, vol. 2015, pp. 18–24, 2015

2015
[13]

A method for stochastic op- timization,

K. D. B. J. Adam et al., “A method for stochastic op- timization,” arXiv preprint arXiv:1412.6980 , vol. 1412, no. 6, 2014

2014 arXiv
[14]

Pytorch: An im- perative style, high-performance deep learning library,

A. Paszke, S. Gross, F. Massa, et al., “Pytorch: An im- perative style, high-performance deep learning library,” Advances in neural information processing systems , vol. 32, 2019

2019
[15]

Evaluation of low-frequency feature restriction and average pooling for acoustic scene classification under unseen-city con- ditions,

T. Kawamura, M. Sera, and N. Ono, “Evaluation of low-frequency feature restriction and average pooling for acoustic scene classification under unseen-city con- ditions,” Technical Report, 2025

2025
[16]

A semi-supervised acoustic scene clas- sification network based on multi-modal information fusion,

J. Yang, H. Liu, L. Shi, L. Gan, H. Nishizaki, and C. S. Leow, “A semi-supervised acoustic scene clas- sification network based on multi-modal information fusion,” Technical Report, 2025

2025

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.

[1] [1]

Acoustic scene classification: Classifying environments from the sounds they produce,

D. Barchiesi, D. Giannoulis, D. Stowell, and M. D. Plumbley, “Acoustic scene classification: Classifying environments from the sounds they produce,” IEEE Signal Processing Magazine , vol. 32, no. 3, pp. 16–34, 2015

work page 2015

[2] [2]

Exploring text-queried sound event detection with audio source separation,

H. Yin, J. Bai, Y . Xiao, et al. , “Exploring text-queried sound event detection with audio source separation,” in IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , IEEE, 2025, pp. 1–5

work page 2025

[3] [3]

Multi-granularity acous- tic information fusion for sound event detection,

H. Yin, J. Chen, J. Bai, et al., “Multi-granularity acous- tic information fusion for sound event detection,” Signal Processing, vol. 227, p. 109 691, 2025

work page 2025

[4] [4]

Description on ieee icme 2024 grand challenge: Semi-supervised acoustic scene classification under domain shift,

J. Bai, M. Wang, H. Liu, et al. , “Description on ieee icme 2024 grand challenge: Semi-supervised acoustic scene classification under domain shift,” arXiv preprint arXiv:2402.02694, 2024

arXiv 2024

[5] [5]

Mamba: Linear-time sequence modeling with selective state spaces,

A. Gu and T. Dao, “Mamba: Linear-time sequence modeling with selective state spaces,” arXiv preprint arXiv:2312.00752, 2023

arXiv 2023

[6] [6]

Attention is all you need,

A. Vaswani, N. Shazeer, N. Parmar, et al., “Attention is all you need,” Advances in neural information process- ing systems, vol. 30, 2017

work page 2017

[7] [7]

Tau urban acoustic scenes 2020 mobile development dataset [data set],

H. Toni, M. Annamaria, and V . Tuomas, “Tau urban acoustic scenes 2020 mobile development dataset [data set],” Zenodo, 2020

work page 2020

[8] [8]

Cochlscene: Acquisition of acoustic scene data using crowdsourcing,

I.-Y . Jeong and J. Park, “Cochlscene: Acquisition of acoustic scene data using crowdsourcing,” in Asia- Pacific Signal and Information Processing Association Annual Summit and Conference, IEEE, 2022, pp. 17–21

work page 2022