Pith. sign in

REVIEW 6 cited by

Masked Autoencoders that Listen

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2207.06405 v3 pith:PY4GW77W submitted 2022-07-13 cs.SD cs.AIcs.LGeess.AS

Masked Autoencoders that Listen

Po-Yao Huang , Hu Xu , Juncheng Li , Alexei Baevski , Michael Auli , Wojciech Galuba , Florian Metze , Christoph Feichtenhofer This is my paper
classification cs.SD cs.AIcs.LGeess.AS
keywords audioaudio-maeautoencodersdecoderencoderlocalmaskedmasking
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved
0 comments
Share X Bluesky LinkedIn Reddit HN
read the original abstract

This paper studies a simple extension of image-based Masked Autoencoders (MAE) to self-supervised representation learning from audio spectrograms. Following the Transformer encoder-decoder design in MAE, our Audio-MAE first encodes audio spectrogram patches with a high masking ratio, feeding only the non-masked tokens through encoder layers. The decoder then re-orders and decodes the encoded context padded with mask tokens, in order to reconstruct the input spectrogram. We find it beneficial to incorporate local window attention in the decoder, as audio spectrograms are highly correlated in local time and frequency bands. We then fine-tune the encoder with a lower masking ratio on target datasets. Empirically, Audio-MAE sets new state-of-the-art performance on six audio and speech classification tasks, outperforming other recent models that use external supervised pre-training. The code and models will be at https://github.com/facebookresearch/AudioMAE.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 6 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. SemanticAudio: Audio Generation and Editing in Semantic Space

    eess.AS 2026-01 conditional novelty 6.0

    SemanticAudio improves text-to-audio alignment by generating a compact semantic plan first with a Flow Matching planner and then rendering acoustic latents from that plan, and it performs training-free audio editing b...

  2. Vision Transformers Need Registers

    cs.CV 2023-09 unverdicted novelty 6.0

    Adding register tokens to Vision Transformers eliminates high-norm background artifacts and raises state-of-the-art performance on dense visual prediction tasks.

  3. CoughPhase-CLR: Designing an acoustics-informed foundation model for coughing sound classification

    cs.SD 2026-06 unverdicted novelty 5.0

    CoughPhase-CLR uses cough physiological phases to build contrastive positive pairs, outperforming random cropping on downstream tasks including COVID-19 detection and COPD classification.

  4. Quality-Diversity Search in Sound Generation: Investigating Innovation Engines for Audio Exploration

    cs.SD 2026-06 unverdicted novelty 5.0

    MAP-Elites with CPPNs, DSP graphs, and a deep classifier produces diverse synthetic sounds across durations and musical/non-musical contexts.

  5. Expectation and Acoustic Neural Network Representations Enhance Music Identification from Brain Activity

    cs.AI 2026-03 unverdicted novelty 5.0

    Separating acoustic and expectation ANN representations as teacher targets improves EEG music identification beyond baselines and seed ensembles.

  6. DINOv2: Learning Robust Visual Features without Supervision

    cs.CV 2023-04 unverdicted novelty 5.0

    Pith review generated a malformed one-line summary.