Pith. sign in

REVIEW 4 major objections 6 minor 57 references

Soft-Constrained Optimization of Latent Space in Variational Autoencoders

T0 review · 4 major / 6 minor · reviewed 2026-07-30 · grok-4.5

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A soft entropy floor on each VAE latent variable raises encoding capacity, and a weight filter then prunes the unused ones down to what a downstream task actually needs.

desk verdict Plausible VAE engineering with an honest limitations section, but the headline numbers sit on a best-of-ten seed protocol the paper’s own citation says can manufacture exactly those gaps. read the letter →

arxiv 2607.23751 v1 pith:45BWN7WO submitted 2026-07-26 cs.LG stat.ML

Ye Shi This is my paper · ORCID
classification cs.LGstat.ML
keywords variationalautoencoderlatentrepresentationconstrainedoptimizationdisentanglemententropyconstraintdimensionalityreductionsoftconstraints
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Variational autoencoders struggle to keep two things at once: latent variables that each carry a lot of information, and a compact, disentangled layout of those variables. Strengthening the usual KL penalty improves disentanglement but kills unused dimensions; weakening it keeps capacity but tangles the factors. This paper reframes VAE training as a soft-constrained problem with an explicit entropy lower bound on each latent variable, showing that a latent code’s entropy upper-bounds the mutual information it can hold about the data’s generative factors. A companion weight filter then drops the lowest-entropy dimensions during downstream training, returning an empirical lower bound on usable dimensionality. On dSprites the constraint lifts aggregate activation by roughly half and tops the FactorVAE scores among β-VAE variants; on MNIST the filter shrinks a ten-dimensional code to two while holding classifier accuracy above 90 percent and converging faster.

What carries the argument

The entropy-based constraint (EC): a soft penalty −γ Σ(δ − H(z_m)) that forces each latent marginal toward a target entropy δ, justified by the identity I(y,z) = H(z) − H(z|x) and the claim that the noise term H(z|x) is already controlled by the shared KL weight.

What would settle it

Train matched models with and without the entropy penalty at fixed β and measure both H(z) and H(z|x); if the noise term moves enough to cancel the gain in H(z), the claimed mutual-information link fails, and the activation and FactorVAE gains should disappear.

Watch

Extended reading notes

Core claim

Imposing a soft lower bound on the entropy of individual latent variables raises their encoding capacity—because that entropy upper-bounds the mutual information with the generative factors—while a weight filter that removes the lowest-entropy dimensions during downstream training yields a feasible lower bound on the latent dimensionality the task actually needs.

Load-bearing premise

The argument assumes that the encoder’s reparameterization noise stays roughly fixed across models, so raising latent entropy is enough to raise the information those latents can carry about the data factors.

Editorial extensions

If this is right

  • Practitioners can monitor and set a per-dimension entropy target instead of tuning a single β that trades capacity against disentanglement.
  • Downstream models can be fed a latent space whose size is discovered on the fly rather than chosen by grid search, cutting classifier input size (here from 10 to 2) while meeting an accuracy floor.
  • Low-entropy discrete factors will tend to merge into one latent, while high-entropy continuous factors will split across several—giving a diagnostic for whether a chosen latent width can represent the factors present in the data.
  • Smaller retained latent spaces lower the cost of every model that consumes them and make the learned factors easier to audit.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same entropy ranking used by the weight filter could serve as a cheap online monitor for over-pruning in any β-VAE-style training run, even without a downstream classifier.
  • If the merging of discrete factors is systematic, an entropy-aware prior that deliberately allocates mixture components to low-entropy labels might improve discrete-factor recovery without extra supervision.
  • Replacing the histogram entropy estimator with a continuous estimator would let the target δ sit on the same scale as the Gaussian prior entropy, tightening the link between the constraint and the mutual-information bound.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper formulates VAE training as a soft-constrained optimization problem with two contributions: (i) an entropy-based constraint (EC) that penalizes Σ(δ − H(z_m)) to raise per-variable latent entropy, motivated by the claim that H(z) upper-bounds I(y,z) with equality in the deterministic-encoder limit; and (ii) a weight-filter that greedily prunes the lowest-entropy latent dimension while a downstream classifier stays above an accuracy threshold. On dSprites the EC raises total LVAS by 43–62% over a vanilla VAE and attains FactorVAE 0.891 vs 0.847 for the β-VAE baseline; on MNIST the filter reduces the classifier's latent input from 10 to 2 dimensions at >90% accuracy with 37% fewer epochs at J=2. The paper also offers a correct and clarifying account of multi-modal aggregate posteriors as Gaussian mixtures over discrete factors, and a nice analysis of the orientation factor's rotational-symmetry degeneracy.

Significance. If the empirical results hold, the EC is a simple, cheap, and easily adopted penalty that gives practitioners a per-dimension quantity to monitor, and the filter is a practical heuristic for sizing latent spaces. The manuscript deserves explicit credit for unusual transparency: §V-D and §V-E disclose the best-of-ten seed protocol, the histogram-estimator bias, the 1/J rescaling confound, and the metric disagreement, and most hyperparameters (β=4, γ=1, δ=1) are fixed a priori rather than tuned. The §V-C observation that dSprites orientation is effectively non-uniform due to shape symmetry is independently useful. However, the headline numbers are single-seed, best-of-ten draws, and the theoretical justification in §III-A contains an error in the treatment of differential entropy, so the contributions currently rest on weaker footing than the abstract suggests.

major comments (4)
  1. [§III-A, Eqs. (10)–(12), Lemma 3.1] The central theoretical claim is incorrect as stated for differential entropy. Lemma 3.1 (H(Y|X)=0 iff Y is determined by X) is a discrete Shannon-entropy result; for a continuous z, a deterministic encoder gives h(z|y) = −∞, not 0, so the 'collapse' of Eq. (10) to I(y,z)=H(z) does not follow, and the claimed deterministic-limit equality is muddled (a deterministic smooth encoder of continuous x yields infinite MI). More importantly, I(y,z) = h(z) − h(z|y) (Eq. 12) implies I ≤ h(z) only when h(z|y) ≥ 0, i.e. σ² ≥ 1/(2πe) ≈ 0.0585; trained VAE posteriors routinely have smaller variances, in which case h(z) does not upper-bound I. The claim is salvageable in the regime where the KL term pins σ² near 1 (as with β=4), but the paper must state and verify that regime, e.g. by reporting the empirical H(z|x) of Eq. (11) for each model. That table would also directly test the load-bearing assumpt
  2. [§III-B, Eqs. (16)–(19); Tables IV–VII EC vs EC+LB rows] As written, the EC term −γΣ(δ − H(z_m)) differs from a pure entropy reward +γΣH(z_m) only by the constant γMδ, so δ has zero gradient and no effect on training. The tables nonetheless report distinct 'EC' and 'EC+LB' rows (e.g. Table VI: total LVAS 10.18 vs 11.58). Either (a) EC and EC+LB use the same loss and seed, in which case the rows should be identical and the differences are unexplained; (b) they use different seeds, in which case the differences are seed noise; or (c) LB is implemented as a hinge penalty max(0, δ−H), in which case Eq. (19) does not describe the model being evaluated and the actual loss must be stated. Relatedly, Eq. (16) constrains only the aggregate Σ(δ−H(z_m)), so individual variables may fall below δ (Table V, min-entropy column, confirms this) — the 'per-variable lower bound' framing should be revised. This must be clarified because two of the five β-family v
  3. [§V-D seed protocol; Tables VI–VII] All headline comparisons are best-of-ten-seed draws selected by reconstruction error, with different seeds for the two families (seed 1 vanilla, seed 10 β). The paper itself cites Locatello et al. [9], whose central result is that dSprites disentanglement scores are dominated by seed and hyperparameter variance. Reconstruction error is not known to correlate with FactorVAE score, so the selected seed's FactorVAE 0.891 vs 0.847 (a 0.044 gap) and the per-label orientation gains are effectively upper-tail draws and are within the seed variance documented in [9]. The 43–62% LVAS gain is large enough that it may survive averaging, but the disentanglement claims likely will not without replication. The authors disclose this honestly (§V-E), but disclosure does not substitute for evidence: the revision needs seed-averaged means and variances (the ten runs already exist) for Tables VI–VII, at mi
  4. [§IV-D, Tables VI–VII; §V-E] The two classifier-based disentanglement metrics disagree about the method (FactorVAE improves under β=4; the β-VAE metric worsens in both families, e.g. 0.5079→0.4172 for β-VAE+EC+LB). The authors argue the asymmetry is 'consistent with the mechanism,' but as they note, two disagreeing metrics cannot adjudicate themselves. Since improved disentanglement is one of the two headline claims, a third, differently constructed measure (e.g. DCI disentanglement/informativeness [7], which is already cited) on the same checkpoints is needed. Separately, the strongest comparisons are against vanilla and β-VAE only; FactorVAE and β-TCVAE are discussed but never run, so 'highest FactorVAE score among the β-VAE variants' should not be read as superiority over methods that target the same trade-off.
minor comments (6)
  1. [Abstract; References; Fig. 4; Eq. (22)] Abstract contains a LaTeX rendering artifact: '\b{eta} \b{eta}-VAE'. Reference [56] has '25rd' for '25th'. Fig. 4 caption: 'The upper figure correspond' → 'corresponds'. Eq. (22) ends with a stray comma.
  2. [§II-D, Eq. (5)] The reparameterization trick is cited to Devroye [49]; the standard citations are Kingma & Welling [2] and Rezende et al. Also, 'm is an arbitrary integer, m ∈ [1,M]' is an odd phrasing for an index.
  3. [§III-C Algorithm 2; §IV-A Table III; §IV-B1] Reproducibility gaps: the histogram bin count K and fixed bin range of Algorithm 2 are never reported, nor is the LVAS activity threshold justified beyond 0.01 (Table VI's 'borderline' sixth dimension at 0.15 shows count sensitivity). The MNIST Adam learning rate of 1e-2 (Table III) is unusually high; confirm it is correct.
  4. [§IV-E, Fig. 9; Algorithm 1] Fig. 9's convergence comparison is acknowledged to conflate dimensionality reduction with the 1/J rescaling; the promised ablation with fixed unit weights (§V-E) would be easy to include and would strengthen the 37%-fewer-epochs claim. Also state the classifier's test (not training) accuracy at J=2, since Algorithm 1 thresholds on training accuracy a.
  5. [§III-A, after Eq. (12)] The claim after Eq. (12) that the averaged KL 'is minimized at σ²=1' is loose: the per-dimension KL is minimized jointly at μ=0, σ²=1. Worth one clause of precision since the argument about the noise term being pinned depends on it.
  6. [Tables VI–VII; Abstract] NALV is reported prominently but the paper itself argues (correctly) that it is a coarse statistic; consider moving NALV to a secondary position in Tables VI–VII rather than the first column, and soften the abstract phrasing that juxtaposes the 0.891 FactorVAE number with the baselines without noting the single-seed protocol.

Circularity Check

1 steps flagged · score 1.0 of 10

No load-bearing circularity: capacity bound is standard information theory; reported gains are on external probes (LVAS, FactorVAE, classifier accuracy), not on the constrained quantity itself.

  1. self definitional [Section IV-C, Tables IV–V (entropy columns)]
    "Every EC variant raised the average latent-variable entropy above its own baseline: from 5.1163 to 5.3573 and 5.2349 in the vanilla family, and from 5.1432 to between 5.1641 and 5.2409 in the β-VAE family."

    Average H(z_m) is exactly the quantity the EC penalty −γ∑(δ−H(z_m)) pushes upward. Reporting that the average rose is verification that the optimizer obeyed the penalty, not an independent prediction. This is minor and non-load-bearing: the abstract and contribution list lead with LVAS, FactorVAE, reconstruction error, and classifier dimensionality, which are external to the constrained functional.

full rationale

The paper's central derivation (Sec. III-A, Eqs. 10–14) is the elementary identity I(y,z)=H(z)−H(z|y) together with the observation that a shared β keeps the reparameterization-noise term near the prior entropy, so raising marginal H(z_m) raises available capacity. That inequality always holds and is not fitted to the dSprites/MNIST outcomes. The EC is then imposed as an ordinary soft penalty (Eq. 19); success is measured by LVAS, β-VAE/FactorVAE metrics, correlation scores, reconstruction error, and downstream classifier accuracy—none of which is defined in terms of the entropy penalty or the filter weights. The weight filter (Alg. 1–3) greedily drops low-entropy dimensions subject to an external accuracy threshold α; the reported J=2 is an empirical output of that procedure, not a quantity forced by construction. Mild bookkeeping tautology exists only in Tables IV–V, where average plug-in entropy rises under an entropy lower-bound (as any successful optimizer would produce); the paper's headline claims do not rest on those rows. Hyperparameters γ and δ are set a priori (or, for EC* only, tuned on reconstruction); that is ordinary practice, not a fitted-input-called-prediction loop for the disentanglement or dimensionality results. No self-citation chain or uniqueness theorem carries the argument. Score 1 reflects only the trivial entropy-under-entropy-constraint verification, not circularity of the main claims.

Assumptions & free parameters 6 free parameters · 6 assumptions · 2 invented entities

The central claims rest on the standard VAE ELBO and reparameterization, an information-theoretic bound linking latent entropy to mutual information under a roughly constant noise term, hand-chosen or reconstruction-tuned multipliers (β, γ, δ), a histogram plug-in entropy estimator, and a greedy accuracy-threshold filter. No new physical entities are postulated; the free parameters and the constant-noise modeling choice are the main extras beyond prior VAE literature.

free parameters (6)
  • γ (EC KKT multiplier) = 1.0 (default); 0.36 (EC*)
    Strength of the entropy penalty; set to 1 a priori for most runs, or 0.36 via Optuna on reconstruction for EC* variants.
  • δ (target entropy lower bound) = 1
    Per-variable entropy target in the soft constraint; chosen as 1 rather than the prior differential entropy ≈1.42, and applied to discrete histogram estimates whose scale differs from differential entropy.
  • β (KL multiplier) = 1 or 4
    Standard β-VAE weight; fixed to 1 or 4 to match prior work, not re-derived.
  • α (classifier accuracy threshold) = 0.9
    Stops latent-dimension reduction in the weight filter; directly determines the reported J=2 on MNIST.
  • LVAS activity threshold = 0.01
    Azm>0.01 defines an 'active' latent variable for NALV counts.
  • histogram binning (K, fixed range)
    Algorithm 2 plug-in entropy depends on bin count and range; offset from differential entropy is acknowledged but K is not swept in reported results.
assumptions (6)
  • domain assumption Standard VAE ELBO with diagonal-Gaussian encoder and reparameterization z=μ+σ⊙ξ, ξ~N(0,1)
    Section II-D; foundation for the loss and for closed-form H(z|x).
  • standard math I(y,z)=H(z)-H(z|y) and H(z|y)=H(z|x) when x=G(y) is deterministic
    Section III-A, Eqs. 10–12; standard information theory plus deterministic generative process.
  • ad hoc to paper For models sharing a common β, the reparameterization noise term varies little, so capacity differences are governed by H(z)
    Stated after Eq. 12 in III-A; not verified when the new γ term is active.
  • domain assumption KL constraint drives total correlation TC(z) toward zero, so raising marginal entropies raises joint H(z)
    Eq. 13 and following text; standard motivation for β-VAE-style penalties.
  • ad hoc to paper Histogram plug-in entropy on fixed bins is a usable surrogate for differential entropy in the constraint and filter ranking
    Algorithm 2; authors note bias and scale offset vs δ.
  • ad hoc to paper Greedy never-reinstate pruning under accuracy threshold α yields a meaningful empirical lower bound on usable latent dimensionality
    Algorithms 1 and 3; authors concede J is feasible not certified minimal.
invented entities (2)
  • Entropy-based constraint (EC) penalty γ Σ(δ-H(z_m))
    purpose: Explicit soft lower bound on per-variable latent entropy inside the VAE objective
    Core methodological proposal in III-B; standard Lagrange penalty form applied to entropy.
  • Entropy weight filter w⊙z with 1/J rescaling
    purpose: Prune low-entropy latent dimensions during downstream classifier training
    Section III-C and Algorithm 1; mechanism for the dimensionality claim.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Soft-Constrained Optimization of Latent Space in Variational Autoencoders." pith.science (2026). https://pith.science/paper/45BWN7WO

@misc{pith2026260723751,
  author       = {Pith},
  title        = {Pith review of: Soft-Constrained Optimization of Latent Space in Variational Autoencoders},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/45BWN7WO}},
  note         = {Machine review of arXiv:2607.23751}
}
read the original abstract

The usefulness of a variational autoencoder (VAE) depends on two properties of its latent space that are hard to obtain together: high encoding capacity in the individual latent variables, and a low-dimensional, disentangled organization of those variables. Weakening the Kullback-Leibler regularization raises capacity but degrades disentanglement, while strengthening it prunes latent variables away entirely. We formulate VAE training as a soft-constrained optimization problem that addresses both. First, we impose an entropy-based constraint (EC) on individual latent variables, showing that the entropy of a latent code upper-bounds the mutual information it carries about the generative factors of the data. Second, we propose a weight-filter method that exploits the slack of the soft constraint to prune low-entropy dimensions during downstream training. On dSprites, the EC raises the aggregate latent-variable activation score by 43-62% over a vanilla VAE, attains the highest FactorVAE score among the \b{eta} \b{eta}-VAE variants (0.891 vs 0.847), and lowers reconstruction error by up to 38%. On MNIST, the weight filter reduces the latent dimensionality supplied to a downstream classifier from ten to two while holding accuracy above 90%, converging in 37% fewer epochs than the same procedure without the EC. We also find that low-entropy discrete factors tend to merge into a single latent variable, whereas high-entropy continuous factors are distributed across several.

Figures

Figures reproduced from arXiv: 2607.23751 by the authors.

Figure 1
Figure 1. VAE structure. The red dashed box demonstrates how [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. A schematic of the optimization of latent-space dimen [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. Comparison of data preprocessing for β-VAE metric and FactorVAE metric. The left flow chart shows the data preparation process for the β-VAE metric, while the right flow chart shows the process for the FactorVAE metric. 4) Correlation-based Metric: In our implementation, we have observed instances where certain labels are occasionally encoded in more than one latent variable. This occurrence may be attributed to eit… view at source ↗
Figures from the paper (8 more)
Figure 5
Figure 5. Figure 5: Vanilla VAE experiment disentanglement scores. The [PITH_FULL_IMAGE:figures/full_fig_p011_5.png]
Figure 4
Figure 4. Figure 4: Individual latent variable activation scores. The upper [PITH_FULL_IMAGE:figures/full_fig_p011_4.png]
Figure 6
Figure 6. Figure 6: β-VAE experiment disentanglement scores. The his￾tograms show the disentanglement score of each label. (a) Vanilla VAE (b) VAE+EC (c) VAE+EC+LB [PITH_FULL_IMAGE:figures/full_fig_p012_6.png]
Figure 8
Figure 8. Figure 8: Cross correlation between latent variables and labeled [PITH_FULL_IMAGE:figures/full_fig_p012_8.png]
Figure 9
Figure 9. Figure 9: MNIST latent-space dimension versus classifier ac [PITH_FULL_IMAGE:figures/full_fig_p013_9.png]
Figure 11
Figure 11. Figure 11: Vanilla VAE latent-variable histograms of the dSprites [PITH_FULL_IMAGE:figures/full_fig_p013_11.png]
Figure 10
Figure 10. Figure 10: Heatmap of normalized latent-variable entropy along [PITH_FULL_IMAGE:figures/full_fig_p013_10.png]
Figure 12
Figure 12. Figure 12: β-VAE latent-variable histograms of the dSprites dataset. The figure setup is identical to [PITH_FULL_IMAGE:figures/full_fig_p014_12.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

57 extracted references · 1 canonical work pages

  1. [9]

    Challenging common assumptions in the unsupervised learning of disentangled representations,

    F. Locatello, S. Bauer, M. Lucic, G. Raetsch, S. Gelly, B. Schölkopf, and O. Bachem, “Challenging common assumptions in the unsupervised learning of disentangled representations,” in International Conference on Machine Learning. PMLR, 2019, pp. 4114–4124

    2019

  2. [7]

    A framework for the quantitative evaluation of disentangled representations,

    C. Eastwood and C. K. Williams, “A framework for the quantitative evaluation of disentangled representations,” in International Conference on Learning Representations , 2018

    2018

  3. [1]

    An introduction to variational autoen- coders,

    D. P. Kingma and M. Welling, “An introduction to variational autoen- coders,” Foundations and Trends® in Machine Learning, vol. 12, no. 4, p. 307–392, Nov 2019

    2019

  4. [2]

    Auto-Encoding Variational Bayes,

    ——, “Auto-Encoding Variational Bayes,” arXiv:1312.6114 [cs, stat] , Dec. 2013

    arXiv 2013

  5. [3]

    Lossy image compres- sion with compressive autoencoders,

    L. Theis, W. Shi, A. Cunningham, and F. Huszár, “Lossy image compres- sion with compressive autoencoders,” arXiv preprint arXiv:1703.00395, 2017

    arXiv 2017

  6. [4]

    Variational autoencoders pursue pca directions (by accident),

    M. Rolinek, D. Zietlow, and G. Martius, “Variational autoencoders pursue pca directions (by accident),” in 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . IEEE, Jun. 2019, p. 12398–12407

    2019

  7. [5]

    Isolating sources of disentanglement in variational autoencoders,

    R. T. Q. Chen, X. Li, R. B. Grosse, and D. K. Duvenaud, “Isolating sources of disentanglement in variational autoencoders,” in Advances in Neural Information Processing Systems , S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett, Eds., vol. 31. Curran Associates, Inc.,

  8. [6]

    Towards a definition of disentangled representations,

    I. Higgins, D. Amos, D. Pfau, S. Racaniere, L. Matthey, D. Rezende, and A. Lerchner, “Towards a definition of disentangled representations,” arXiv preprint arXiv:1812.02230 , no. arXiv:1812.02230, Dec 2018, arXiv:1812.02230 [cs, stat]. [Online]. Available: http://arxiv.org/abs/ 1812.02230

    arXiv 2018

Show all 57 references
  1. [8]

    Multi-level varia- tional autoencoder: Learning disentangled representations from grouped observations,

    D. Bouchacourt, R. Tomioka, and S. Nowozin, “Multi-level varia- tional autoencoder: Learning disentangled representations from grouped observations,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 32, no. 1, 2018

    2018

  2. [10]

    A Multimodal Anomaly Detector for Robot-Assisted Feeding Using an LSTM-Based Variational Autoen- coder,

    D. Park, Y . Hoshi, and C. C. Kemp, “A Multimodal Anomaly Detector for Robot-Assisted Feeding Using an LSTM-Based Variational Autoen- coder,” IEEE Robotics and Automation Letters , vol. 3, no. 3, pp. 1544– 1551, Jul. 2018

    2018

  3. [11]

    Anomaly detection through latent space restoration using vector-quantized variational autoencoders,

    S. N. Marimont and G. Tarroni, “Anomaly detection through latent space restoration using vector-quantized variational autoencoders,” Dec. 2020

    2020

  4. [12]

    Latent Space Oddity: On the Curvature of Deep Generative Models,

    G. Arvanitidis, L. K. Hansen, and S. Hauberg, “Latent Space Oddity: On the Curvature of Deep Generative Models,” Dec. 2021

    2021

  5. [13]

    Importance Weighted Autoencoders,

    Y . Burda, R. Grosse, and R. Salakhutdinov, “Importance Weighted Autoencoders,” Nov. 2016

    2016

  6. [14]

    Tackling Over- pruning in Variational Autoencoders,

    S. Yeung, A. Kannan, Y . Dauphin, and L. Fei-Fei, “Tackling Over- pruning in Variational Autoencoders,” Aug. 2017

    2017

  7. [15]

    Sparsity in Variational Autoencoders,

    A. Asperti, “Sparsity in Variational Autoencoders,” Feb. 2019

    2019

  8. [16]

    Wasserstein Auto-Encoders,

    I. Tolstikhin, O. Bousquet, S. Gelly, and B. Schoelkopf, “Wasserstein Auto-Encoders,” Dec. 2019

    2019

  9. [17]

    Epitomic Variational Graph Autoencoder,

    R. A. Khan, M. U. Anwaar, and M. Kleinsteuber, “Epitomic Variational Graph Autoencoder,” Apr. 2020

    2020

  10. [18]

    Beta-V AE: Learning Basic Visual Concepts with a Constrained Variational Framework,

    I. Higgins, L. Matthey, A. Pal, C. P. Burgess, X. Glorot, M. Botvinick, S. Mohamed, and A. Lerchner, “Beta-V AE: Learning Basic Visual Concepts with a Constrained Variational Framework,” in International Conference on Learning Representations , Nov. 2016

    2016

  11. [19]

    Generative adversarial nets,

    I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. C. Courville, and Y . Bengio, “Generative adversarial nets,” in NIPS, 2014. 15

    2014

  12. [20]

    A Survey on Generative Diffusion Model,

    H. Cao, C. Tan, Z. Gao, G. Chen, P.-A. Heng, and S. Z. Li, “A Survey on Generative Diffusion Model,” Dec. 2022

    2022

  13. [21]

    A Survey on Variational Autoencoders from a Green AI Perspective,

    A. Asperti, D. Evangelista, and E. Loli Piccolomini, “A Survey on Variational Autoencoders from a Green AI Perspective,” SN Computer Science, vol. 2, no. 4, p. 301, Jul. 2021

    2021

  14. [22]

    Balancing Reconstruction Error and Kullback-Leibler Divergence in Variational Autoencoders,

    A. Asperti and M. Trentin, “Balancing Reconstruction Error and Kullback-Leibler Divergence in Variational Autoencoders,” IEEE Ac- cess, vol. 8, pp. 199 440–199 448, 2020

    2020

  15. [23]

    The Autoencoding Variational Autoencoder,

    T. Cemgil, S. Ghaisas, K. Dvijotham, S. Gowal, and P. Kohli, “The Autoencoding Variational Autoencoder,” in Advances in Neural Infor- mation Processing Systems, vol. 33. Curran Associates, Inc., 2020, pp. 15 077–15 087

    2020

  16. [24]

    Consistency Regularization for Variational Auto-Encoders,

    S. Sinha and A. B. Dieng, “Consistency Regularization for Variational Auto-Encoders,” Jun. 2022

    2022

  17. [25]

    Infogan: Interpretable representation learning by information maximizing generative adversarial nets,

    X. Chen, Y . Duan, R. Houthooft, J. Schulman, I. Sutskever, and P. Abbeel, “Infogan: Interpretable representation learning by information maximizing generative adversarial nets,” arXiv:1606.03657 [cs, stat] , Jun. 2016, arXiv: 1606.03657

    2016 arXiv

  18. [26]

    Disentangling by factorising,

    H. Kim and A. Mnih, “Disentangling by factorising,” in International Conference on Machine Learning . PMLR, 2018, pp. 2649–2658

    2018

  19. [27]

    When Is Unsupervised Dis- entanglement Possible?

    D. Horan, E. Richardson, and Y . Weiss, “When Is Unsupervised Dis- entanglement Possible?” in Advances in Neural Information Processing Systems, vol. 34. Curran Associates, Inc., 2021, pp. 5150–5161

    2021

  20. [28]

    Encouraging Disentangled and Convex Representation with Controllable Interpolation Regularization,

    Y . Ge, Z. Xu, Y . Xiao, G. Xin, Y . Pang, and L. Itti, “Encouraging Disentangled and Convex Representation with Controllable Interpolation Regularization,” in 2023 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) . Waikoloa, HI, USA: IEEE, Jan. 2023, pp. 4750–4758

    2023

  21. [29]

    Varia- tional Autoencoders and Nonlinear ICA: A Unifying Framework,

    I. Khemakhem, D. P. Kingma, R. P. Monti, and A. Hyvärinen, “Varia- tional Autoencoders and Nonlinear ICA: A Unifying Framework,” Dec. 2020

    2020

  22. [30]

    GD-V AEs: Geometric Dynamic Varia- tional Autoencoders for Learning Nonlinear Dynamics and Dimension Reductions,

    R. Lopez and P. J. Atzberger, “GD-V AEs: Geometric Dynamic Varia- tional Autoencoders for Learning Nonlinear Dynamics and Dimension Reductions,” Dec. 2022

    2022

  23. [31]

    Nonlinear Dimension Reduction by PDF Estimation,

    P. M. Baggenstoss and S. Kay, “Nonlinear Dimension Reduction by PDF Estimation,” IEEE Transactions on Signal Processing , vol. 70, pp. 1493–1505, 2022

    2022

  24. [32]

    A Variational Autoencoder- Based Dimensionality Reduction Technique for Generation Forecasting in Cyber-Physical Smart Grids,

    D. Kaur, S. N. Islam, and M. A. Mahmud, “A Variational Autoencoder- Based Dimensionality Reduction Technique for Generation Forecasting in Cyber-Physical Smart Grids,” in 2021 IEEE International Conference on Communications Workshops (ICC Workshops) , Jun. 2021, pp. 1–6

    2021

  25. [33]

    Connections with robust PCA and the role of emergent sparsity in variational autoencoder models,

    B. Dai, Y . Wang, J. Aston, G. Hua, and D. Wipf, “Connections with robust PCA and the role of emergent sparsity in variational autoencoder models,” Journal of Machine Learning Research , vol. 19, no. 41, pp. 1–42, 2018

    2018

  26. [34]

    Sample-efficient optimization in the latent space of deep generative models via weighted retraining,

    A. Tripp, E. Daxberger, and J. M. Hernández-Lobato, “Sample-efficient optimization in the latent space of deep generative models via weighted retraining,” arXiv:2006.09191 [cs, stat] , Jun. 2020, arXiv: 2006.09191

    2006 arXiv

  27. [35]

    Optimizing the latent space of generative networks,

    P. Bojanowski, A. Joulin, D. Lopez-Paz, and A. Szlam, “Optimizing the latent space of generative networks,” arXiv preprint arXiv:1707.05776 , 2017

    2017 arXiv

  28. [36]

    Maskaae: Latent space optimization for adversarial auto-encoders,

    A. Mondal, S. P. Chowdhury, A. Jayendran, H. Asnani, P. Singla, and P. A. P, “Maskaae: Latent space optimization for adversarial auto-encoders,” in Conference on Uncertainty in Artificial Intelligence . PMLR, Aug. 2020, p. 689–698

    2020

  29. [37]

    Goodfellow, Y

    I. Goodfellow, Y . Bengio, A. Courville, and Y . Bengio, Deep learning. MIT press Cambridge, 2016, vol. 1, no. 2

    2016

  30. [38]

    Explicitly imposing constraints in deep networks via conditional gradients gives improved generalization and faster convergence,

    S. N. Ravi, T. Dinh, V . S. Lokhande, and V . Singh, “Explicitly imposing constraints in deep networks via conditional gradients gives improved generalization and faster convergence,” in Proceedings of the AAAI Conference on Artificial Intelligence , vol. 33, no. 01, 2019, pp....

    2019

  31. [39]

    Constrained convolutional neural networks for weakly supervised segmentation,

    D. Pathak, P. Krahenbuhl, and T. Darrell, “Constrained convolutional neural networks for weakly supervised segmentation,” in Proceedings of the IEEE international conference on computer vision , 2015, pp. 1796– 1804

    2015

  32. [40]

    Constrained policy opti- mization,

    J. Achiam, D. Held, A. Tamar, and P. Abbeel, “Constrained policy opti- mization,” in International Conference on Machine Learning . PMLR, 2017, pp. 22–31

    2017

  33. [41]

    Spectral norm regularization for improving the generalizability of deep learning,

    Y . Yoshida and T. Miyato, “Spectral norm regularization for improving the generalizability of deep learning,” arXiv preprint arXiv:1705.10941, no. arXiv:1705.10941, May 2017, arXiv:1705.10941 [cs, stat]. [Online]. Available: http://arxiv.org/abs/1705.10941

    2017 arXiv

  34. [42]

    Deep learning of constrained autoen- coders for enhanced understanding of data,

    B. O. Ayinde and J. M. Zurada, “Deep learning of constrained autoen- coders for enhanced understanding of data,” IEEE transactions on neural networks and learning systems , vol. 29, no. 9, pp. 3969–3979, 2017

    2017

  35. [43]

    l1 -norm batch normalization for efficient training of deep neural networks,

    S. Wu, G. Li, L. Deng, L. Liu, D. Wu, Y . Xie, and L. Shi, “ l1 -norm batch normalization for efficient training of deep neural networks,” IEEE Transactions on Neural Networks and Learning Systems , vol. 30, no. 7, p. 2043–2051, Jul 2019

  36. [44]

    Tau-fpl: Tolerance-constrained learning in linear time,

    A. Zhang, N. Li, J. Pu, J. Wang, J. Yan, and H. Zha, “Tau-fpl: Tolerance-constrained learning in linear time,” in Proceedings of the AAAI Conference on Artificial Intelligence , vol. 32, no. 1, 2018

    2018

  37. [45]

    Constrained deep learning using conditional gradient and applications in computer vision,

    S. N. Ravi, T. Dinh, V . Lokhande, and V . Singh, “Constrained deep learning using conditional gradient and applications in computer vision,” 2018

    2018

  38. [46]

    Application of constrained learning in making deep networks more transparent, regularized, and biologically plausible,

    H. A. Ghiassirad, M. A. Shoorehdeli, and F. Farivar, “Application of constrained learning in making deep networks more transparent, regularized, and biologically plausible,” Engineering Applications of Artificial Intelligence, vol. 85, pp. 421–428, 2019

    2019

  39. [47]

    End-to- end constrained optimization learning: A survey,

    J. Kotary, F. Fioretto, P. Van Hentenryck, and B. Wilder, “End-to- end constrained optimization learning: A survey,” arXiv:2103.16378 [cs], Mar 2021, arXiv: 2103.16378. [Online]. Available: http: //arxiv.org/abs/2103.16378

    2021 arXiv

  40. [48]

    COIL: Constrained optimization in learned latent space: Learning representations for valid solutions,

    P. J. Bentley, S. L. Lim, A. Gaier, and L. Tran, “COIL: Constrained optimization in learned latent space: Learning representations for valid solutions,” in Proceedings of the Genetic and Evolutionary Computation Conference Companion , ser. GECCO ’22. New York, NY , USA: Associ...

    2022

  41. [49]

    Sample-based non-uniform random variate generation,

    L. Devroye, “Sample-based non-uniform random variate generation,” in Proceedings of the 18th conference on Winter simulation , ser. WSC ’86. Association for Computing Machinery, Dec 1986, p. 260–265. [Online]. Available: https://doi.org/10.1145/318242.318443

    1986

  42. [50]

    Karush, Minima of Functions of Several Variables with Inequalities as Side Conditions

    W. Karush, Minima of Functions of Several Variables with Inequalities as Side Conditions . Basel: Springer, 2014, p. 217–245. [Online]. Available: https://doi.org/10.1007/978-3-0348-0439-4_10

    2014 doi

  43. [51]

    Nonlinear programming,

    H. W. Kuhn and A. W. Tucker, “Nonlinear programming,” Proceedings of the Second Berkeley Symposium on Mathematical Statistics and Probability, vol. 2, p. 481–493, Jan 1951

    1951

  44. [52]

    Understanding disentangling in β-vae,

    C. P. Burgess, I. Higgins, A. Pal, L. Matthey, N. Watters, G. Desjardins, and A. Lerchner, “Understanding disentangling in β-vae,” arXiv preprint arXiv:1804.03599, 2018

    2018 arXiv

  45. [53]

    Entropy, Relative Entropy, and Mutual Information,

    T. M. Cover and J. A. Thomas, “Entropy, Relative Entropy, and Mutual Information,” in Elements of Information Theory . John Wiley & Sons, Ltd, 2005, ch. 2, pp. 13–55

    2005

  46. [54]

    dsprites: Disentanglement testing sprites dataset,

    L. Matthey, I. Higgins, D. Hassabis, and A. Lerch- ner, “dsprites: Disentanglement testing sprites dataset,” https://github.com/deepmind/dsprites-dataset/, 2017

    2017

  47. [55]

    The mnist database of handwritten digit images for machine learning research [best of the web],

    L. Deng, “The mnist database of handwritten digit images for machine learning research [best of the web],” IEEE Signal Processing Magazine, vol. 29, no. 6, pp. 141–142, 2012

    2012

  48. [56]

    Optuna: A next- generation hyperparameter optimization framework,

    T. Akiba, S. Sano, T. Yanase, T. Ohta, and M. Koyama, “Optuna: A next- generation hyperparameter optimization framework,” in Proceedings of the 25rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining , 2019. 16

    2019

  49. [2018]

    Available: https://proceedings.neurips.cc/paper/2018/ file/1ee3dfcd8a0645a25a35977997223d22-Paper.pdf

    [Online]. Available: https://proceedings.neurips.cc/paper/2018/ file/1ee3dfcd8a0645a25a35977997223d22-Paper.pdf

    2018

Pith tools

Reviewed July 30, 2026 · model on record in the stance chip above.