Denoising Multi-Beta VAE: Representation Learning for Disentanglement and Generation

Anshuk Uppal; Chieh-Hsin Lai; Yuhta Takida; Yuki Mitsufuji

REVIEW 4 major objections 5 minor 1 cited by

Denoising Multi-Beta VAE: Representation Learning for Disentanglement and Generation

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Treating beta-VAE information loss as a noising process, a nonlinear diffusion model can reverse it, giving disentangled codes and sharp images from the same model.

desk verdict Strong empirical results and a promising multi-beta idea are undercut by a load-bearing error in the diffusion derivation. read the letter →

arxiv 2507.06613 v1 pith:TG2HMW3B submitted 2025-07-09 cs.LG cs.AIcs.CV

Anshuk Uppal , Yuhta Takida , Chieh-Hsin Lai , Yuki Mitsufuji This is my paper · ORCID

classification cs.LGcs.AIcs.CV

keywords disentangledrepresentationlearningvariationalautoencoderbeta-VAEdiffusionmodelsnonlinearimagegenerationlatentmanipulationinformationbottleneck

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to dissolve the $\beta$-VAE trade-off, in which strong disentanglement of latent codes blurs reconstructions. It trains one VAE whose encoder and decoder are conditioned on $\beta$, so a single model spans a spectrum from lossy, highly disentangled codes at large $\beta$ to near-lossless codes at $\beta=0$, and then trains a nonlinear diffusion model that walks any high-$\beta$ latent back toward $\beta=0$ before decoding. The intended payoff is that disentangled representation learning and sharp, high-quality generation become two phases of one pipeline rather than competing objectives. On CelebA the combined model reports a disentanglement metric of TAD 0.378 with FID 17.9, beating the dedicated disentangled-diffusion baseline DisDiff on both axes.

What carries the argument

The load-bearing object is the $\beta$-conditioned multi-level VAE: using $\beta$ both as a network input and as the interpolation weight in Eq. (4) gives a family of latent spaces ordered by information content, with learnable isotropic variances $\sigma^2_\beta$ that double as the diffusion noise schedule. The second mechanism is the nonlinear diffusion model whose reverse mean in Eq. (17) adds the learned encoder-difference predictor $\hat{\Delta}_\psi$ to the usual noise-prediction term; this extra term is what carries a high-$\beta$ latent back to the near-lossless $\beta=0$ space. The design move that ties them together is identifying $\beta$ with diffusion time $t$, so one trained model covers both representation learning and generation.

What would settle it

Derive the exact reverse conditional $q(z_{t-\tau}\mid z_t,x)$ from Eq. (14) and compare it term-by-term with Eq. (16); a dimensional or algebraic mismatch would mean the model is trained to predict a posterior that never arises from the forward process, so the reported TAD and FID gains would not establish that information loss is being reversed.

Watch

Extended reading notes

Core claim

The paper's central claim is that the information lost as $\beta$ increases can be modeled as a noising process and then reversed by diffusion. A $\beta$-conditioned VAE trained with the rescaled objective $\mathcal{L}_\beta = (B-\beta)\,\mathbb{E}_{q_\phi(z_\beta|x)}[\log p_\theta(x|z_\beta)] - \beta\,D_{\mathrm{KL}}(q_\phi(z_\beta|x)\|p(z))$ produces latent spaces $\mathcal{Z}_\beta$ whose retained information shrinks monotonically as $\beta$ grows; the paper proves the extremes, $\sigma^2_\beta \to 0$ as $\beta\to 0$ and zero mutual information at $\beta=1$. The accompanying nonlinear diffusion model treats time and $\beta$ as the same coordinate and learns two predictors, the noise $\hat{\epsilon}_\psi$ and the encoder-difference term $\hat{\Delta}_\psi$, so its sampling loop can transport a latent from large $\beta$ back to $\beta=0$, where the decoder produces sharp images; starting from pure noise performs unconditional generation. The experiments report TAD 0.378 and FID 17.9 on CelebA, surpassing DisDiff, and competitive FID on CelebA-HQ, FFHQ, and LSUN-Bedrooms, with latent edits that remain consistent across inputs and across $\beta$ values.

Load-bearing premise

The whole denoising argument stands on the unproven reverse-time mean in Eq. (16); if that formula does not actually invert the forward noising process of Eq. (14), the training signal does not justify the claimed denoising.

Editorial extensions

If this is right

One $\beta$-conditioned model replaces many separate $\beta$-VAE training runs, since any $\beta$ in $[0,B]$ is available after a single training pass.
Reconstructions and image edits stay sharp even for highly disentangled high-$\beta$ codes, because the denoiser first maps the latent back toward $\beta=0$ before the decoder sees it.
The framework is a standalone generative model: sample $z_T \sim \mathcal{N}(0,I)$ and run the learned denoising loop to produce images.
On the reported benchmarks, disentanglement no longer costs generation quality: the CelebA result improves on DisDiff in both TAD and FID.
Because the latent spaces are smooth in $\beta$, the same latent-space direction produces consistent attribute changes across different input images.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Beyond the paper: because different generative factors peak at different $\beta$ values, one could choose the $\beta$ that maximizes mutual information for the attribute to be edited, apply the edit there, and then denoise to $\beta=0$ for a sharp result.
Beyond the paper: the sensitivity of training quality to the learned $\sigma_\beta$ schedule suggests that jointly learning that schedule with the VAE could remove dataset-specific tuning and make the method more portable.
Beyond the paper: the same 'denoise across an ordered family of bottlenecked spaces' idea could transfer to other rate-distortion families, such as compression autoencoders, whenever representations are ordered by information loss.
Beyond the paper: switching from the DDPM-style sampler to a DDIM-style one, which the paper lists as future work, would reveal whether the 1000-step sampling cost is essential or merely an implementation choice.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Short version: the paper has a good instinct and impressive numbers, but the central diffusion derivation does not hold up. Eq. (16) is wrong as written, and the sampler used in practice does not correspond to the stated reverse process.

What's actually new: the multi-beta VAE is a clean extension of beta-VAE. Training one model over a continuous beta interval with a rescaled ELBO gives a spectrum of latents from lossless to disentangled. That part is useful and worth building on. The second stage is the more interesting claim: a nonlinear latent-space diffuser that maps high-beta latents to low-beta ones for sharp generation. Using the learned encoder as a time-indexed noising process is genuinely novel, and the empirical numbers on CelebA, FFHQ, and LSUN-Bedrooms are strong if they reproduce.

But the math doesn't back the diffusion claim. The forward process in Eq. (14) is Gaussian with mean z_{t-tau} + f(x,t) - f(x,t-tau). The reverse should come from Gaussian conditioning on (z_t, x). The correct posterior mean is (sigma_{t-tau}^2/sigma_t^2) z_t + f(x,t-tau) - (sigma_{t-tau}^2/sigma_t^2) f(x,t). Eq. (16) instead has a term with x in pixel space added to latent vectors, and a different combination of the f terms. That's dimensionally inconsistent and, more importantly, not what you'd derive from Eq. (14). Worse, Algorithm 2 uses mu_t = z_t - sigma_t epsilon_hat and mu_{t-tau} = mu_t - Delta_hat, which doesn't match Eq. (16) regardless. So the forward/reverse pair isn't justified, and the training loss (19) isn't validated as any posterior-matching objective. That's a load-bearing gap, not a typo.

There's also a sign error in the proof of Prop 3.1: Eq. (26) asserts H(p(x'|x)) <= H(beta I), but the conditional has covariance beta I plus a positive term, so the inequality should go the other way. The proposition is almost a restatement of the loss and is not hard to prove, but the proof as written is wrong. And on the toy benchmarks they sweep beta on the test set to select the best latent, which inflates the disentanglement numbers. That's a common sin in this literature, but it still undermines the comparisons.

To give credit: the paper is well-organized, covers the right baselines, and is transparent about limitations. The multi-beta idea alone might be publishable if the diffuser were removed or fixed.

I'd send this to a serious referee—the problem is important and the empirical claims are strong—but I'd expect major revision or rejection as is. If the authors derive the correct reverse process and make the sampler consistent with it, or explicitly recast the sampler as a heuristic with empirical justification, the paper could come back solid. As submitted, the theory does not support the method.

Referee Report

4 major / 5 minor

Summary. The paper proposes a two-stage generative framework for disentangled representation learning. In the first stage, a conditional multi-beta VAE is trained with a rescaled ELBO over the range β∈[0,B], producing a spectrum of latent representations that interpolate between lossless (β=0) and fully regularized (β=B) regimes. In the second stage, a non-linear diffusion model is introduced to reverse the information loss, denoising a high-β latent back to a low-β latent and thereby enabling sharp reconstructions and unconditional generation. The framework is evaluated on disentanglement benchmarks (CelebA TAD/FID; Cars3D, Shapes3D, MPI3D with MIG/DCI) and generation benchmarks (CelebA-HQ, FFHQ, LSUN-Bedrooms with FID/precision/recall). The central theoretical claim is that the diffusion model in Section 4.2 reverses the encoding process defined by the forward conditional in Eq. (14).

Significance. The idea of training a single VAE across a continuum of β values and then using a diffusion-style model to move along that spectrum is original, and if it worked it would address a well-known trade-off between disentanglement and generation quality. The empirical results are strong: the method reports the best TAD and FID on CelebA among the compared methods and competitive FID on three large-scale image datasets. The paper also provides pseudocode for the training and sampling algorithms, which supports reproducibility of the proposed procedure. However, the central theoretical derivation contains a fundamental mathematical error: the reverse posterior in Eq. (16) is dimensionally inconsistent, and the sampling algorithm in Algorithm 2 is not connected to the asserted reverse process. Because these issues concern the core mechanism of the proposed method, the empirical results cannot, in their present form, be attributed to a valid generative model that reverses the noising process in Eq. (14).

major comments (4)

[Section 4.2, Eq. (16)] The reverse posterior mean in Eq. (16) is dimensionally inconsistent and is not derived from the forward process in Eq. (14). The term (σ_{t|t−τ}^2/σ_t^2) x adds a pixel-space vector x ∈ R^D to latent-space quantities z_t, f(x,t−τ), and f(x,t) ∈ R^d, which is impossible when d < D. A correct Gaussian conditioning of Eq. (14) with marginals q_φ(z_s|x) = N(f(x,s), σ_s^2 I) would yield a posterior mean involving f(x,t−τ) and f(x,t) with specific coefficients, not the expression in Eq. (16). Since Eq. (16) motivates the parameterization in Eq. (17) and the training loss in Eq. (19), the proposed non-linear diffusion model is not mathematically grounded as a reversal of the noising process.
[Section 4.2, Algorithm 2] Algorithm 2 is not a discretization of Eqs. (15)–(17). At sampling time the conditioning image x is unavailable, yet Eq. (16) depends on x explicitly. The algorithm instead computes µ_t = z_t − σ_t ε̂ and µ_{t−T/N} = µ_t − Δ̂, which contains neither the coefficient σ_{t−τ}^2/σ_t^2 nor any term resembling the image-dependent contributions in Eq. (16). No derivation is provided linking this update to the reverse conditional, so the sampling procedure appears to be a heuristic that is disjoint from the theoretical development. Consequently, the generation results in Table 3 and Figure 6 cannot be claimed to follow from the proposed forward-reverse pair.
[Appendix B, Propositions 3.1 and 3.2; Section 3.2] The proof of Proposition 3.1 conflates the KL coefficient β with the decoder variance s_β^2. In Eq. (26) of Appendix B.1, the conditional entropy of p_{θ,φ}(x'|x) is bounded using H(βI), but the decoder is defined with covariance s_β^2 I in Eq. (2); the claim that H(βI) → −∞ as β → 0 therefore does not apply to the actual model. The proof also invokes Theorem 3 of Takida et al. (2022) and Theorem 4 of Dai et al. (2019) as black boxes without stating the conditions under which they apply, leaving the proposition undemonstrated. The related monotonicity assertion σ_β < σ_β' for 0 ≤ β < β' ≤ 1 in Section 3.2, which is load-bearing for treating σ_t as a noise schedule, is stated without proof or specification of conditions.
[Section 6.1.2 and Appendix C.1.3] The disentanglement results are reported at a per-dataset best value of β chosen by sweeping over all β values (e.g., β = 285/500 for Cars3D), while baselines are reported as single models. This introduces an unaccounted selection bias. The same issue applies to the choice between the two noise schedules Sched 1 and Sched 2 in Table 6, which is made per dataset. A fair evaluation would report performance at a fixed β for all datasets or the average over the sweep, and would specify the noise-schedule selection protocol a priori. Without this, the claim of "comparable or surpassing" disentanglement performance in Table 2 is not fully supported.

minor comments (5)

[Figure 1 caption, page 3] The caption uses "non-zero b" instead of "non-zero β"; please fix the typo and ensure β is used consistently in the figure.
[Section 4.2, page 6] The phrase "our approach enables both disentanglement and high-quality generation in principle" in the introduction and contribution list is hedged; either remove "in principle" or provide a concrete statement of the conditions under which the claim holds.
[Section 3.1, page 4] The sentence "Notably, L_β when B = 1, with β = 0 and 0.5, corresponds to the objective functions for a plain autoencoder and a VAE, respectively, without considering the scaling factors" is confusing, since at β = 0.5 Eq. (4) gives 0.5 × reconstruction + 0.5 × KL, which is not the standard VAE objective up to an irrelevant scaling; please clarify the intended correspondence.
[Table 3, page 9] In Table 3, the column headers "Prec.↑Recall↑" should be formatted as two separate column headers to avoid ambiguity.
[Appendix C.1.3, page 17] The statement that "none of our models work well for a schedule that grows sub-linearly for t < 0.5" is an unexplained empirical observation; please provide supporting evidence or remove the claim.

Circularity Check

2 steps flagged · score 4.0 of 10

Partial circularity: Proposition 3.2 restates the β=1 objective and toy-dataset β is swept on the reported metric; the main CelebA/generation results are independent.

self definitional [Section 3.2, Eq. (4) and Proposition 3.2]
"when β = 1, the objective function (4) reduces to the KL regularization term, causing qϕ(z|x; β = 1) collapse to the prior distribution p(z). Consequently, the latent space, i.e., Z1, no longer retains any information about the input x, as demonstrated in the following proposition."

With B = 1, Eq. (4) at β = 1 is exactly L1 = −E[D_KL(qϕ(z|x) ∥ p(z))], so the reconstruction term is zeroed by definition. The 'information loss at β = 1' conclusion of Proposition 3.2 is therefore a restatement of the chosen objective, not an independent prediction: the proof only routes the already-penalized KL through the data-processing inequality. Presenting this as a demonstrated mechanism is a self-definitional step, although it is an auxiliary motivation rather than the source of the main empirical numbers.
fitted input called prediction [Appendix C.1.2 (toy disentanglement benchmarks, Table 2)]
"We perform a sweep over all β values in our model to identify the optimal latent representation for each dataset. Specifically, for Cars3D, we achieve the highest score at β = 285/500; for Shapes3D, at β = 210/500; and for MPI3D, at β = 280/500."

The reported 'Ours' MIG/DCI entries in Table 2 are obtained by selecting β to maximize those same MIG/DCI scores on the same toy datasets. The table therefore reports the maximum of the metric over the tuning grid, not the score of a fixed, pre-specified model; the disentanglement comparison is partially forced by the selection procedure. The paper discloses the sweep, but the result is a fitted score rather than an independent prediction.

full rationale

The central advertised results are not circular in the load-bearing sense. Table 1 (CelebA TAD/FID) and Table 3 (FID/precision/recall on CelebA-HQ, FFHQ, LSUN-Bedrooms) compare against external baselines on held-out metrics, and no fitted target is renamed as a generation prediction. The self-citations to Takida et al. (2022) for Proposition 3.1 and 3.2 are minor: the authors overlap, but the appendix includes proofs and the propositions are motivation rather than the source of the empirical claims. The main circularity-adjacent issues are (i) Proposition 3.2, where 'information loss at β = 1' is already written into Eq. (4) by zeroing the reconstruction weight, and (ii) the toy-dataset beta sweep, where β is selected on the same MIG/DCI metric subsequently reported. A separate, non-circularity correctness concern is that Eq. (16) is asserted without derivation from the forward process Eq. (14) and mixes pixel-space x with latent-space vectors; this is a derivation gap, not a circularity, but it should be fixed before the diffusion model can be considered justified as a reverse process.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central framework relies on the empirical beta-disentanglement relationship and on the validity of the nonlinear diffusion posterior, which is not derived. The evaluation beta per dataset is fitted to the test metric.

free parameters (3)

evaluation beta per dataset = e.g., 285/500 (Cars3D), 210/500 (Shapes3D), 280/500 (MPI3D)
The authors sweep beta on each toy dataset and select the value maximizing the reported disentanglement metric; this is fitting a hyperparameter to the test metric.
noise schedule (Sched 1 / Sched 2) = not given in the paper; shapes shown in Fig. 3
The sigma_beta schedule is learned from the VAE's training under a chosen beta schedule; the authors report that the diffusion model is sensitive to this schedule and that only certain schedules work. The exact functional forms are only in shared code.
loss weighting w(t) = not stated
The weighting function in Eq. (18)-(19) is not specified, and it affects the diffusion training.

assumptions (4)

domain assumption Larger beta values induce more disentangled latent representations and higher latent variance (sigma_beta increases with beta).
Section 3.2 states this as the motivation, citing previous observations (Takida et al. 2022); it is not derived from Eq. (4) in the paper.
domain assumption The learned conditional VAE's variance sigma_beta can serve as a valid diffusion noise schedule for the nonlinear diffusion model.
Section 3.2, last paragraph: 'we purposely combine the model ... so that learnt sigma_beta parallel a typical noising process in diffusion models (beta = t).' This is an assumption about the learned schedule.
ad hoc to paper The reverse posterior Eq. (15)-(16) follows from the forward process Eq. (14).
No derivation is provided, and the equation as written is dimensionally inconsistent (image x added to latent vectors), so this is an unsupported assumption.
standard math Theorem 4 in Dai and Wipf (2019) and Theorem 3 in Takida et al. (2022) are correct and applicable to the conditional beta-VAE.
Used in Appendix B.1 and B.2 for the proofs of Prop 3.1 and 3.2.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Denoising Multi-Beta VAE: Representation Learning for Disentanglement and Generation." pith.science (2026). https://pith.science/paper/TG2HMW3B

@misc{pith2026250706613,
  author       = {Pith},
  title        = {Pith review of: Denoising Multi-Beta VAE: Representation Learning for Disentanglement and Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TG2HMW3B}},
  note         = {Machine review of arXiv:2507.06613}
}

abstract

Disentangled and interpretable latent representations in generative models typically come at the cost of generation quality. The $\beta$-VAE framework introduces a hyperparameter $\beta$ to balance disentanglement and reconstruction quality, where setting $\beta > 1$ introduces an information bottleneck that favors disentanglement over sharp, accurate reconstructions. To address this trade-off, we propose a novel generative modeling framework that leverages a range of $\beta$ values to learn multiple corresponding latent representations. First, we obtain a slew of representations by training a single variational autoencoder (VAE), with a new loss function that controls the information retained in each latent representation such that the higher $\beta$ value prioritize disentanglement over reconstruction fidelity. We then, introduce a non-linear diffusion model that smoothly transitions latent representations corresponding to different $\beta$ values. This model denoises towards less disentangled and more informative representations, ultimately leading to (almost) lossless representations, enabling sharp reconstructions. Furthermore, our model supports sample generation without input images, functioning as a standalone generative model. We evaluate our framework in terms of both disentanglement and generation quality. Additionally, we observe smooth transitions in the latent spaces with respect to changes in $\beta$, facilitating consistent manipulation of generated outputs.

Figures

Figures reproduced from arXiv: 2507.06613 by the authors.

**Figure 1.** Our framework for achieving both disentanglement and generation. Our approach embeds β values as time conditioning in our newly designed nonlinear diffusion model enabling both effective disentanglement and high-quality generation. (a-b) Directly decoding zb at non-zero b results in blurred images; however, applying the denoiser before decoding yields clear images. (c-d) The denoiser also improves the quality of man… view at source ↗

**Figure 2.** Smoothness of latent space in β (t): We provide evidence for the smoothness of the learned representations by generating samples from latent spaces with various values of β. Notably, latent representations obtained by smaller β values tend to produce images closer to ground truth because they retain more information. Burgess, Christopher P., Irina Higgins, Arka Pal, Loic Matthey, Nick Watters, Guillaume Desjardins, … view at source ↗

**Figure 3.** Examples of preferable noise schedules in our framework. These noise schedules have [PITH_FULL_IMAGE:figures/full_fig_p018_3.png] view at source ↗

Figures from the paper (5 more)

**Figure 4.** Figure 4: Variation of MIG and MI: We analyze the variation in MIG and MI scores of generative factors across different β values on the Cars3D dataset (Reed et al., 2015) to quantitatively track disentanglement in the latent space. Notably, the highest MIG scores for each factor…

**Figure 5.** Figure 5: This figure presents examples of how our multi- [PITH_FULL_IMAGE:figures/full_fig_p021_5.png]

**Figure 6.** Figure 6: Samples generated by models trained on CelebA-HQ (Karras et al., [PITH_FULL_IMAGE:figures/full_fig_p022_6.png]

**Figure 7.** Figure 7: Interpolation in latent space: We interpolate pairs of images (shown in the leftmost and rightmost columns) by applying slerp to the latent spaces. D.5 Exploring the Spectrum of Learned Latents We investigate the properties of latent spaces by interpolating pairs of im…

**Figure 8.** Figure 8: This figure contains more examples of editing of facial attributes of a person. In the first [PITH_FULL_IMAGE:figures/full_fig_p024_8.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

CoVAE: Consistency Training of Variational Autoencoders
stat.ML 2025-07 conditional novelty 6.0 of 10

CoVAE trains a time-dependent VAE with a consistency loss so one or few decoder passes generate images, reaching FID 5.62 on MNIST and 11.69 on CIFAR-10 with adversarial loss, without a learned prior.

Reference graph

Works this paper leans on

15 extracted references · 13 canonical work pages · cited by 1 Pith paper

[1]

Multi-Rate V AE: Train Once, Get the Full Rate-Distortion Curve

Bae, Juhan, Michael R. Zhang, Michael Ruan, Eric Wang, So Hasegawa, Jimmy Ba, and Roger Baker Grosse (2023). “Multi-Rate V AE: Train Once, Get the Full Rate-Distortion Curve”. In:The Eleventh International Conference on Learning Representations. 9 =50/1000 =250/1000 =450/1000 =650/1000 =750/1000 =850/1000 =900/1000 =950/1000=0/1000 (reconstruc�on) Ground ...

work page 2023
[2]

simple diffusion: End-to- end diffusion for high resolution images

Hoogeboom, Emiel, Jonathan Heek, and Tim Salimans (23–29 Jul 2023). “simple diffusion: End-to- end diffusion for high resolution images”. In: Proceedings of the 40th International Conference on Machine Learning. Ed. by Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett. V ol

work page 2023
[6]

and thus sample for 1000 steps. 20 0 300 500 550 600 650 700 750 800 900 999 Time/Beta values 0 300 500 550 600 650 700 750 800 900 999 Time/Beta values 0 300 500 550 600 650 700 750 800 900 999 Time/Beta values Figure 5: This figure presents examples of how our multi-β V AE reconstructs images across different datasets as β is varied. The top panel shows...

work page 2015
[7]

Both algorithms require a trained multi-β V AE and the corresponding nonlinear diffusion model to generate high-fidelity images

We begin by producing an interpolated encoding at a time step t using an interpolation coefficient α to control the mixing between z1 and z2. Both algorithms require a trained multi-β V AE and the corresponding nonlinear diffusion model to generate high-fidelity images. D Additional Results This section presents additional results from applying our method...

work page 2022
[9]

Preventing oversmoothing in V AE via generalized variance parameterization

Proceedings of Machine Learning Research. PMLR, pp. 8655–8664. Takida, Yuhta, Wei-Hsiang Liao, Chieh-Hsin Lai, Toshimitsu Uesaka, Shusuke Takahashi, and Yuki Mitsufuji (2022). “Preventing oversmoothing in V AE via generalized variance parameterization”. In: Neurocomputing 509, pp. 137–156. Vahdat, Arash, Karsten Kreis, and Jan Kautz (2021). “Score-based G...

arXiv 2022
[10]

Since pD(x) is expressed as a sum of Dirac delta functions δ(x − xi) for i = 1 ,

The two assumptions in this proposition are reasonable in practice. Since pD(x) is expressed as a sum of Dirac delta functions δ(x − xi) for i = 1 , . . . , M, where each xi represents a set of bounded pixel values, its covariance is finite. Furthermore, because fϕ is implemented within a deep neural network framework, it is expected to have strong repres...

work page 2022
[11]

Since the additive noise for the reparameterization trick follows a normal distribution N (0, Id), the covariance matrix satisfies Σϕ,σ2 = σ2I + Σe ϕ

In relation to (ii), let V and L represent constants corresponding to the maximum singular value of the covariance of pD(x) and Lipschitz constant of fϕ(x). Since the additive noise for the reparameterization trick follows a normal distribution N (0, Id), the covariance matrix satisfies Σϕ,σ2 = σ2I + Σe ϕ. Therefore, we have det(Σϕ,σ2 ) = det(σ2I + Σe ϕ) ...

work page 2022
[12]

and x′ ∼ pθ(x′|z; β = 1), respectively. Applying the data processing inequality to the Markov chain x → z → x′ yields: I(x; z) ≥ I(x; x′), (28) Additionally, the mutual information I(x; z) can be rewritten using its definition as follows: I(x; z) = DKL(pD(x)qϕ(z|x)||pD(x)qϕ(z)) = EpD(x)qϕ(z|x)[log qϕ(z|x) − log qϕ(z)] = EpD(x) DKL(qϕ(z|x)||p(z)) − DKL(qϕ(...

work page 2021

Show all 15 references

[28]

Learning Disentangled Repre- sentation by Exploiting Pretrained Generative Models: A Contrastive Learning View

Curran Associates, Inc. Ren, Xuanchi, Tao Yang, Yuwang Wang, and Wenjun Zeng (2022). “Learning Disentangled Repre- sentation by Exploiting Pretrained Generative Models: A Contrastive Learning View”. In:ICLR. Richards, Ryan J and Austen M Groener (2022). “Conditional β-V AE for...

2022 arXiv
[30]

beta-V AE: Learning Basic Visual Concepts 10 with a Constrained Variational Framework

Curran Associates, Inc. Higgins, Irina, Loïc Matthey, Arka Pal, Christopher P. Burgess, Xavier Glorot, Matthew M. Botvinick, Shakir Mohamed, and Alexander Lerchner (2016). “beta-V AE: Learning Basic Visual Concepts 10 with a Constrained Variational Framework”. In:International...

2016
[32]

The encoder consists of four layers with [64, 64, 128, 256] channels, respectively, and we apply GroupNorm after every convolutional layer, followed by aSigmoid non-linearity

The encoder and decoder are symmetric networks that are fully convolutional. The encoder consists of four layers with [64, 64, 128, 256] channels, respectively, and we apply GroupNorm after every convolutional layer, followed by aSigmoid non-linearity. For training these model...

2021
[34]

A U-Net Based Discriminator for Generative Adversarial Networks

Curran Associates, Inc., pp. 17480– 17492. Schonfeld, Edgar, Bernt Schiele, and Anna Khoreva (June 2020). “A U-Net Based Discriminator for Generative Adversarial Networks”. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Shao, Huaj...

2020
[97]

Adversarial Autoencoders

Proceedings of Machine Learning Research. PMLR, pp. 4114–4124. Makhzani, Alireza, Jonathon Shlens, Navdeep Jaitly, and Ian J. Goodfellow (2015). “Adversarial Autoencoders”. In: ArXiv abs/1511.05644. Mathieu, Emile, Tom Rainforth, N. Siddharth, and Yee Whye Teh (2018). “Disenta...

2015 arXiv
[119]

Deep learning face attributes in the wild

Proceedings of Machine Learning Research. PMLR, pp. 6127–6139. Liu, Ziwei, Ping Luo, Xiaogang Wang, and Xiaoou Tang (2015). “Deep learning face attributes in the wild”. In: Proceedings of the IEEE international conference on computer vision, pp. 3730–3738. Locatello, Francesco...

2015
[202]

Progressive Growing of GANs for Improved Quality, Stability, and Variation

Proceedings of Machine Learning Research. PMLR, pp. 13213–13232. Hoogeboom, Emiel, Thomas Mensink, Jonathan Heek, Kay Lamerigts, Ruiqi Gao, and Tim Salimans (2024). Simpler Diffusion (SiD2): 1.5 FID on ImageNet512 with pixel-space diffusion. Karras, Tero, Timo Aila, Samuli Lai...

2024 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.

[1] [1]

Multi-Rate V AE: Train Once, Get the Full Rate-Distortion Curve

Bae, Juhan, Michael R. Zhang, Michael Ruan, Eric Wang, So Hasegawa, Jimmy Ba, and Roger Baker Grosse (2023). “Multi-Rate V AE: Train Once, Get the Full Rate-Distortion Curve”. In:The Eleventh International Conference on Learning Representations. 9 =50/1000 =250/1000 =450/1000 =650/1000 =750/1000 =850/1000 =900/1000 =950/1000=0/1000 (reconstruc�on) Ground ...

work page 2023

[2] [2]

simple diffusion: End-to- end diffusion for high resolution images

Hoogeboom, Emiel, Jonathan Heek, and Tim Salimans (23–29 Jul 2023). “simple diffusion: End-to- end diffusion for high resolution images”. In: Proceedings of the 40th International Conference on Machine Learning. Ed. by Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett. V ol

work page 2023

[3] [6]

and thus sample for 1000 steps. 20 0 300 500 550 600 650 700 750 800 900 999 Time/Beta values 0 300 500 550 600 650 700 750 800 900 999 Time/Beta values 0 300 500 550 600 650 700 750 800 900 999 Time/Beta values Figure 5: This figure presents examples of how our multi-β V AE reconstructs images across different datasets as β is varied. The top panel shows...

work page 2015

[4] [7]

Both algorithms require a trained multi-β V AE and the corresponding nonlinear diffusion model to generate high-fidelity images

We begin by producing an interpolated encoding at a time step t using an interpolation coefficient α to control the mixing between z1 and z2. Both algorithms require a trained multi-β V AE and the corresponding nonlinear diffusion model to generate high-fidelity images. D Additional Results This section presents additional results from applying our method...

work page 2022

[5] [9]

Preventing oversmoothing in V AE via generalized variance parameterization

Proceedings of Machine Learning Research. PMLR, pp. 8655–8664. Takida, Yuhta, Wei-Hsiang Liao, Chieh-Hsin Lai, Toshimitsu Uesaka, Shusuke Takahashi, and Yuki Mitsufuji (2022). “Preventing oversmoothing in V AE via generalized variance parameterization”. In: Neurocomputing 509, pp. 137–156. Vahdat, Arash, Karsten Kreis, and Jan Kautz (2021). “Score-based G...

arXiv 2022

[6] [10]

Since pD(x) is expressed as a sum of Dirac delta functions δ(x − xi) for i = 1 ,

The two assumptions in this proposition are reasonable in practice. Since pD(x) is expressed as a sum of Dirac delta functions δ(x − xi) for i = 1 , . . . , M, where each xi represents a set of bounded pixel values, its covariance is finite. Furthermore, because fϕ is implemented within a deep neural network framework, it is expected to have strong repres...

work page 2022

[7] [11]

Since the additive noise for the reparameterization trick follows a normal distribution N (0, Id), the covariance matrix satisfies Σϕ,σ2 = σ2I + Σe ϕ

In relation to (ii), let V and L represent constants corresponding to the maximum singular value of the covariance of pD(x) and Lipschitz constant of fϕ(x). Since the additive noise for the reparameterization trick follows a normal distribution N (0, Id), the covariance matrix satisfies Σϕ,σ2 = σ2I + Σe ϕ. Therefore, we have det(Σϕ,σ2 ) = det(σ2I + Σe ϕ) ...

work page 2022

[8] [12]

and x′ ∼ pθ(x′|z; β = 1), respectively. Applying the data processing inequality to the Markov chain x → z → x′ yields: I(x; z) ≥ I(x; x′), (28) Additionally, the mutual information I(x; z) can be rewritten using its definition as follows: I(x; z) = DKL(pD(x)qϕ(z|x)||pD(x)qϕ(z)) = EpD(x)qϕ(z|x)[log qϕ(z|x) − log qϕ(z)] = EpD(x) DKL(qϕ(z|x)||p(z)) − DKL(qϕ(...

work page 2021