pith. sign in

arxiv: 2205.07547 · v2 · pith:26RQW4MNnew · submitted 2022-05-16 · 💻 cs.LG · cs.CV

SQ-VAE: Variational Bayes on Discrete Representation with Self-annealed Stochastic Quantization

Yuhta Takida , Takashi Shibuya , WeiHsiang Liao , Chieh-Hsin Lai , Junki Ohmura , Toshimitsu Uesaka , Naoki Murata , Shusuke Takahashi
show 2 more authors
Toshiyuki Kumakura Yuki Mitsufuji
This is my paper
classification 💻 cs.LG cs.CV
keywords sq-vaequantizationcodebookstochastictrainingvariationalvq-vaeautoencoder
0
0 comments X
read the original abstract

One noted issue of vector-quantized variational autoencoder (VQ-VAE) is that the learned discrete representation uses only a fraction of the full capacity of the codebook, also known as codebook collapse. We hypothesize that the training scheme of VQ-VAE, which involves some carefully designed heuristics, underlies this issue. In this paper, we propose a new training scheme that extends the standard VAE via novel stochastic dequantization and quantization, called stochastically quantized variational autoencoder (SQ-VAE). In SQ-VAE, we observe a trend that the quantization is stochastic at the initial stage of the training but gradually converges toward a deterministic quantization, which we call self-annealing. Our experiments show that SQ-VAE improves codebook utilization without using common heuristics. Furthermore, we empirically show that SQ-VAE is superior to VAE and VQ-VAE in vision- and speech-related tasks.

This paper has not been read by Pith yet.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 7 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Differentiable Semantic ID for Generative Recommendation

    cs.IR 2026-01 unverdicted novelty 7.0

    DIGER makes semantic IDs in generative recommendation differentiable via Gumbel noise and decay schedules, yielding consistent gains on public datasets by aligning indexing and recommendation losses.

  2. Finite Scalar Quantization: VQ-VAE Made Simple

    cs.CV 2023-09 conditional novelty 7.0

    Finite scalar quantization simplifies VQ-VAE latents by independently rounding a few dimensions to fixed levels, producing an equivalent-sized implicit codebook with competitive performance and no collapse.

  3. ArcVQ-VAE: A Spherical Vector Quantization Framework with ArcCosine Additive Margin

    cs.CV 2026-05 unverdicted novelty 6.0

    ArcVQ-VAE constrains VQ-VAE codebook vectors inside a time-dependent ball and adds angular margin loss to increase separability and codebook utilization.

  4. Continuous Latent Diffusion Language Model

    cs.CL 2026-05 unverdicted novelty 6.0

    Cola DLM proposes a hierarchical latent diffusion model that learns a text-to-latent mapping, fits a global semantic prior in continuous space with a block-causal DiT, and performs conditional decoding, establishing l...

  5. Two-Dimensional Quantization for Geometry-Aware Audio Coding

    cs.SD 2025-12 unverdicted novelty 6.0

    Q2D2 uses 2D geometric grid projections to quantize feature pairs in neural audio codecs, yielding implicit codebooks that improve efficiency and utilization over RVQ, VQ, and FSQ while maintaining reconstruction quality.

  6. ArcVQ-VAE: A Spherical Vector Quantization Framework with ArcCosine Additive Margin

    cs.CV 2026-05 unverdicted novelty 5.0

    ArcVQ-VAE adds spherical angular-margin regularization consisting of ball-bounded norms and arc-cosine margin loss to improve codebook utilization in VQ-VAE for image tasks.

  7. Improving 3D Gaussian Splatting Compression by Scene-Adaptive Lattice Vector Quantization

    cs.CV 2025-09 unverdicted novelty 5.0

    Scene-adaptive lattice vector quantization improves rate-distortion performance of 3DGS compression over uniform scalar quantization while adding little overhead and supporting multiple bit rates from one trained model.