REVIEW 4 major objections 4 minor 2 cited by
Data Augmentation with Variational Autoencoder for Imbalanced Dataset
T0 review · 4 major / 4 minor · reviewed 2026-08-11 · deepseek-v4-flash
Pith's one-line read The paper claims that a variational autoencoder trained with an inverse-density weighted loss and generating new points by smoothed bootstrap in its latent space—the DAVID algorithm—outperforms the initial training sample and existing…
desk verdict DAVID's method is sensible and the simulation is decent, but the real-data claim rests on a metric the method is built to optimize. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The object that carries the argument is the DAVID generator: a $\beta$-VAE for regression whose target-reconstruction term is weighted by $\omega_i = 1/\hat{f}_Y(y_i)^\alpha$ (inverse empirical density of the target, estimated by kernel density with Silverman or Scott bandwidth), followed by a smoothed bootstrap over the encoder's latent means $\mu_i$. The smoothed bootstrap draws a seed $\mu_i$ with probability $\omega_i$ and adds Gaussian noise $K_{H_n}(z^* - \mu_i)$ with $H_n = \eta \cdot \mathrm{Var}(\mu)$, where $\eta$ follows Silverman's or Scott's rule of thumb. This replaces the VAE's native sampling $z_i = \mu_i + \sigma_i \varepsilon$, which is unreliable for rare values because $\sigma_i$ is poorly estimated. The smoothed bootstrap uses the neighborhood structure of the latent space, and the VAE's regularity makes that neighborhood meaningful; this is what allows the non-parametric generation of rare target values.
What would settle it
On an imbalanced-regression benchmark, train DAVID and a control that uses the same balanced-loss VAE but generates from the native per-observation Gaussians $\mathcal{N}(\mu_i, \sigma_i)$ instead of the smoothed bootstrap, with identical seeds and $\alpha$; if the control matches DAVID's wMSE, the smoothed bootstrap is not the active ingredient. Alternatively, compute the reconstruction error of DAVID's generated rare-value points through the encoder-decoder: if the error is no better than random latent draws, the generator is producing off-manifold samples.
Extended reading notes
Core claim
The central discovery is that the failure of VAEs on imbalanced regression is not primarily a reconstruction problem but a generation problem: the per-observation latent variance $\sigma_i$ for rare values is poorly estimated and too large, so sampling from $\mathcal{N}(\mu_i, \sigma_i)$ pushes generated points off the rare-value manifold. DAVID replaces this with a joint generator $g_{z^*}(z^*|\mu) = \sum_i \omega_i K_{H_n}(z^* - \mu_i)$, a Gaussian-mixture smoothed bootstrap over the latent means $\mu_i$ with bandwidth $H_n$ from Silverman's or Scott's rule, and seed weights $\omega_i = 1/\hat{f}_Y(y_i)^\alpha$. Because the VAE's latent space is regular (continuous and complete), kernel density estimation is valid there—unlike in the original data space or in a plain autoencoder's latent space. The authors show each component matters: the balanced loss improves over the vanilla $\beta$-VAE, the smoothed bootstrap improves over natural VAE generation, and combined (kBVAEw, which is DAVID) they beat the baseline and competitors on weighted MSE.
Load-bearing premise
The method's gains depend on the VAE learning a latent space that is regular and faithful enough that kernel-smoothed neighborhoods around the latent means $\mu_i$ correspond to valid, on-manifold rare-value samples; if the VAE is poorly trained, the generated points are off-manifold and augmentation can worsen regression, as the paper itself notes.
Editorial extensions
If this is right
- DAVID lowers weighted MSE on the four real benchmark datasets (bank8fm, abalone, boston, NO2), achieving the best mean wMSE among all compared methods.
- Standard synthetic generators not designed for imbalanced regression (TVAE, CTGAN, CopulaGAN) degrade performance relative to the initial sample, so they should not be used for this task without adaptation.
- Generating in the latent space of a regular VAE is more effective than generating in the original data space (kTrain) or in PCA or kernel-PCA latent spaces.
- The balanced loss alone (BVAEw) improves over the vanilla VAE, and the smoothed bootstrap alone (kBVAE) is comparable to natural VAE generation, so the two modifications are complementary.
- Because the method is a preprocessing step, it works with any downstream regressor, including the white-box models preferred in insurance and finance.
Reading between the lines
- A testable extension is to adapt DAVID's weighting scheme to classification by replacing the continuous target density with class-frequency weights, then applying the smoothed bootstrap to generate minority-class latent samples without binning the target.
- The success of the smoothed bootstrap suggests that VAE variance estimates are the bottleneck for rare-value generation; one could directly regularize $\sigma_i$ (e.g., shrink large variances) and test whether the gap between natural VAE generation and smoothed bootstrap closes.
- The method's dependence on the VAE's latent regularity implies that the generator's performance can serve as a diagnostic for latent-space quality: if wMSE degrades, the encoder has likely not learned a continuous manifold.
- Since the paper averages results over ten models via H2O AutoML, the gains are already model-averaged; a further test would isolate which base learners benefit most from DAVID augmentation.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes DAVID, a data augmentation method for imbalanced regression (IR) on tabular data, which combines a β-VAE with a balanced loss and a smoothed bootstrap over the latent means. The balanced loss weights observations by the inverse of the estimated target density, and the smoothed bootstrap generates new latent points from kernel density estimates centered on the latent means, with seeds drawn according to the same inverse-density weights. The method is evaluated on one simulated dataset and four real IR benchmark datasets, comparing against several baselines and state-of-the-art methods. The authors report that DAVID outperforms the initial training sample and the competitors, especially on the real datasets where only the self-defined wMSE metric is reported.
Significance. If the reported gains hold, the method is a useful contribution to the under-studied area of imbalanced regression, offering a principled way to adapt VAEs to IR without discretizing the target variable. The paper's strengths include a clearly described algorithm, a step-by-step ablation on the simulation, the use of multiple learning algorithms via H2O AutoML, and publicly available code on GitHub. The simulation results do include standard MSE/MAE/MAPE metrics, which is commendable. However, the central claim of superiority on real datasets rests on a custom wMSE metric that is aligned with the training objective, and the real-data evaluation lacks standard error metrics and statistical significance tests. The paper also acknowledges a key limitation—the method fails if the VAE is not functioning properly—but does not provide diagnostics to delineate when it works. These gaps currently prevent full verification of the central claim.
major comments (4)
- [§5, Table 2] The central claim that DAVID 'gives better results than the initial training sample and the state-of-the-art approaches' is supported in Section 5 only by the self-defined wMSE, whose weights ω_i = 1/f_Y(y_i)^α (Eq. 2) coincide with the inverse-density weights used in the training loss (Eq. 1) and in the smoothed-bootstrap seed drawing (Section 3.4). Because the metric and the training objective share the same weighting, the reported wMSE improvement is partly a direct consequence of the method's design; to establish the claim, the authors should report standard MSE/MAE (as done in the simulation, Table 1) on the real datasets, ideally together with a metric that is not aligned with the training loss.
- [§5, Tables 2 and 1] No statistical significance testing is provided for the 10-fold comparisons. For example, bank8FM baseline wMSE is 1.87 ± 0.19 versus DAVID's 1.68 ± 0.13, and abalone 6.84 ± 0.29 versus 5.52 ± 0.22; these differences are small relative to the reported spreads, so it is unclear whether the improvement is systematic. Paired tests (e.g., Wilcoxon signed-rank across folds) or confidence intervals should be added for both tables; otherwise the 'better results' claim is not robust.
- [§4.2] Two state-of-the-art methods are missing from the comparison: ADASYN was removed after the authors observed poor results and high computation time, and VIR (Wang and Wang, 2024) was not compared because its code is no longer accessible. The VIR exclusion is particularly relevant because VIR is the closest existing VAE-based imbalanced regression method, and the paper's claim of outperforming state-of-the-art VAE approaches cannot be verified without it. At minimum, the authors should discuss the expected comparison and report any partial results; the ADASYN exclusion should be documented with its quantitative results in an appendix rather than reported only qualitatively.
- [§6, Discussion] The paper explicitly states 'the method would not work if the VAE is not functioning properly,' yet no diagnostic of VAE quality (e.g., reconstruction error on rare values, or latent-space uniformity checks) is reported for the simulations or real datasets. Since the method's validity depends on the latent space accurately reflecting the data, the empirical section should include such diagnostics or a sensitivity analysis over β_KL and α to demonstrate the regime in which DAVID is effective. This is not a fatal flaw given the acknowledgment, but it is needed to bound the method's scope.
minor comments (4)
- [§5, Table 2] Table 2 contains a row 'kTrain' that is not defined in the protocol list in §4.2; presumably it refers to smoothed bootstrap applied in the original data space (called CSB in Table 1), but the naming should be consistent.
- [§4.2, Table 1] The label 'OVAE' is used inconsistently for what should be '0VAE' (zero-KL VAE); the text and tables should use the same label.
- [§3.2, Eq. (1)] In Eq. (1), the weighting term is written as 'bf (Y )α', but the notation is not formally defined; it should be f̂_Y(y)^α to match Eq. (2).
- [§4.2 and Appendix A] The protocol text states that 70% of the data is allocated to the training set, while Appendix A specifies a 60-40% train-test split; this inconsistency should be reconciled.
Circularity Check
The real-data superiority claim is evaluated with the same inverse-density weights that the method is built to optimize, making the headline wMSE gain partly tautological.
-
fitted input called prediction
[Section 3.2, Eq. (1); Section 3.3, Eq. (2); Section 4.3, wMSE definition; Section 5, Table 2]
""we propose to weigh the observations during the training phase by the inverse of the empirical density of Y" ... L(θ, ϕ,x, y) = β_x E_q[log pθ(x|z)] − β_KL D_KL(qθ(z|x)||pθ(z)) + β_y / bf(Y)^α E_q[log pθ(y|z)] (1) ... "ω_i := 1/\hat f_Y(y_i)^α (2)" ... "Our weighted MSE wMSE(Y, Ŷ) := 1/n Σ ω_i (y_i − ŷ_i)^2 with ω_i defined as in 2" ... "DA VID ('kBV AEw') gives better results than the initial training sample and the state-of-the-art approaches.""
The same inverse-target-density weights ω_i are fitted from the training data and then inserted into three places: the VAE objective (Eq. 1 weights the y-reconstruction by β_y / f(Y)^α), the seed drawing of the smoothed bootstrap (Sec. 3.3: "the first step is represented by the drawing weight ω_i"), and the evaluation metric wMSE (Sec. 4.3, using "ω_i defined as in 2"). Thus the headline real-data result in Table 2 is a comparison on the very metric that DAVID is constructed to improve: rebalancing the loss and the generated sample by 1/f_Y^α will, other things equal, reduce that same 1/f_Y^α-weighted test error.
full rationale
The paper's derivation chain is otherwise self-contained: the VAE loss, the smoothed-bootstrap generator, and the DAVID algorithm are defined independently of the empirical claim, and no load-bearing conclusion is imported from the authors' earlier work (Stocksieker et al. 2023 appears only as related work, not as a justification of the method). The Discussion also candidly limits the method to cases where the VAE works, saying "the method would not work if the VAE is not functioning properly," which is an external modeling assumption rather than a circular step. The circularity concern is specifically evaluative: Eq. (2) defines ω_i from the inverse density of the target, Eq. (1) uses those weights in the training loss, the smoothed bootstrap uses them for seed drawing, and Section 4.3 defines wMSE with "ω_i defined as in 2." Since Table 2 reports only wMSE for the real datasets, the central claim that "DA VID ... gives better results than the initial training sample and the state-of-the-art approaches" is measured with a metric that the method is explicitly designed to optimize. This makes the headline real-data gain partly tautological, though not fully forced because the downstream regressors are not directly minimizing wMSE. The simulation results with standard MSE/MAE provide some independent support, so the overall circularity is partial rather than total.
Assumptions & free parameters
free parameters (5)
- alpha (exponent for inverse density weights) =
1
- beta_y (weight of target reconstruction in loss) =
10
- beta_KL (weight of KL divergence) =
1e-6
- noise scaling parameter for smoothed bootstrap =
0.1
- latent dimension q =
int(p/10)+1
assumptions (5)
- standard math VAE evidence lower bound (ELBO) formulation and reparameterization trick are valid for the proposed loss.
- standard math Kernel density estimation with Silverman's or Scott's rule-of-thumb bandwidth is consistent.
- standard math Smoothed bootstrap is consistent in the Mallows metric (De Martini and Rapallo 2008; Falk and Reiss 1989).
- domain assumption The VAE's latent space is regular enough (continuity and completeness) for kernel smoothing to generate valid samples.
- domain assumption Rare target values are those with low empirical density f_hat(y), and upweighting them during VAE training improves reconstruction and generation of rares.
Cite this review
Pith. "Pith review of Data Augmentation with Variational Autoencoder for Imbalanced Dataset." pith.science (2026). https://pith.science/paper/MDZGAIT7
@misc{pith2026241207039,
author = {Pith},
title = {Pith review of: Data Augmentation with Variational Autoencoder for Imbalanced Dataset},
year = {2026},
howpublished = {\url{https://pith.science/paper/MDZGAIT7}},
note = {Machine review of arXiv:2412.07039}
}
read the original abstract
Learning from an imbalanced distribution presents a major challenge in predictive modeling, as it generally leads to a reduction in the performance of standard algorithms. Various approaches exist to address this issue, but many of them concern classification problems, with a limited focus on regression. In this paper, we introduce a novel method aimed at enhancing learning on tabular data in the Imbalanced Regression (IR) framework, which remains a significant problem. We propose to use variational autoencoders (VAE) which are known as a powerful tool for synthetic data generation, offering an interesting approach to modeling and capturing latent representations of complex distributions. However, VAEs can be inefficient when dealing with IR. Therefore, we develop a novel approach for generating data, combining VAE with a smoothed bootstrap, specifically designed to address the challenges of IR. We numerically investigate the scope of this method by comparing it against its competitors on simulations and datasets known for IR.
Figures
Forward citations
Cited by 2 Pith papers
-
Taming Data Challenges in ML-based Security Tasks Using Generative AI
Generative AI data augmentation, especially Nimai's sample-conditioned synthesis, improves several security classifiers and speeds drift recovery, but fails on tasks with noisy or overlapping labels.
-
CARTGen-IR: Synthetic Tabular Data Generation for Imbalanced Regression
A CART-based synthetic sampler with rarity-weighted resampling achieves state-of-the-art competitive results for imbalanced regression without target thresholds.
Reference graph
Works this paper leans on
-
[1]
Ai, Q., Wang, P., He, L., Wen, L., Pan, L., and Xu, Z. (2023). Generative oversampling for imbalanced data via majority-guided vae. In International Conference on Artificial Intelligence and Statistics , pages 3315--3330. PMLR
work page 2023
-
[2]
Branco, P., Ribeiro, R. P., and Torgo, L. (2016a). Ubl: an r package for utility-based learning. arXiv preprint arXiv:1604.08079
arXiv 2016
-
[3]
Branco, P., Torgo, L., and Ribeiro, R. P. (2016b). A survey of predictive modeling on imbalanced domains. ACM computing surveys (CSUR) , 49(2):1--50
work page 2016
-
[4]
Branco, P., Torgo, L., and Ribeiro, R. P. (2017). Smogn: a pre-processing approach for imbalanced regression. In First international workshop on learning with imbalanced domains: Theory and applications , pages 36--50. PMLR
work page 2017
-
[5]
Branco, P., Torgo, L., and Ribeiro, R. P. (2018). Rebagg: Resampled bagging for imbalanced regression. In Second International Workshop on Learning with Imbalanced Domains: Theory and Applications , pages 67--81. PMLR
work page 2018
-
[6]
Branco, P., Torgo, L., and Ribeiro, R. P. (2019). Pre-processing approaches for imbalanced distributions in regression. Neurocomputing , 343:76--99
work page 2019
-
[7]
Camacho, L., Douzas, G., and Bacao, F. (2022). Geometric smote for regression. Expert Systems with Applications , page 116387
work page 2022
-
[8]
Smote: Synthetic minority over-sampling technique
Chawla, Bowyer, Hall, and Kegelmeyer (2002). Smote: Synthetic minority over-sampling technique. Journal of Artificial Intelligence Research , 16:321--357
work page 2002
Show all 45 references
-
[9]
and Young, G
De Angelis, D. and Young, G. A. (1992). Smoothing the bootstrap. International Statistical Review/Revue Internationale de Statistique , pages 45--56
1992
-
[10]
and Rapallo, F
De Martini, D. and Rapallo, F. (2008). On multivariate smoothed bootstrap consistency. Journal of statistical planning and inference , 138(6):1828--1835
2008
-
[11]
Ding, Y., Jia, M., Zhuang, J., and Ding, P. (2022). Deep imbalanced regression using cost-sensitive learning and deep feature transfer for bearing remaining useful life estimation. Applied Soft Computing , 127:109271
2022
-
[12]
and Reiss, R.-D
Falk, M. and Reiss, R.-D. (1989). Weak convergence of smoothed and nonsmoothed bootstrap quantile estimates. The Annals of Probability , pages 362--371
1989
-
[13]
C., Krawczyk, B., and Herrera, F
Fern \'a ndez, A., Garc \' a, S., Galar, M., Prati, R. C., Krawczyk, B., and Herrera, F. (2018a). Learning from imbalanced data sets , volume 10. Springer
2018
-
[14]
Fern \'a ndez, A., Garcia, S., Herrera, F., and Chawla, N. V. (2018b). Smote for learning from imbalanced data: progress and challenges, marking the 15-year anniversary. Journal of artificial intelligence research , 61:863--905
2018
-
[15]
Gong, Y., Mori, G., and Tung, F. (2022). Ranksim: Ranking similarity regularization for deep imbalanced regression. arXiv preprint arXiv:2205.15236
2022 arXiv
-
[16]
Haixiang, G., Yijing, L., Shang, J., Mingyun, G., Yuanyue, H., and Bing, G. (2017). Learning from class-imbalanced data: Review of methods and applications. Expert systems with applications , 73:220--239
2017
-
[17]
J., and Romano, J
Hall, P., DiCiccio, T. J., and Romano, J. P. (1989). On smoothing and the bootstrap. The Annals of Statistics , pages 692--704
1989
-
[18]
A., and Li, S
He, H., Bai, Y., Garcia, E. A., and Li, S. (2008). Adasyn: Adaptive synthetic sampling approach for imbalanced learning. In 2008 IEEE international joint conference on neural networks (IEEE world congress on computational intelligence) , pages 1322--1328. IEEE
2008
-
[19]
and Ma, Y
He, H. and Ma, Y. (2013). Imbalanced learning: foundations, algorithms, and applications . John Wiley & Sons
2013
-
[20]
Keramati, M., Meng, L., and Evans, R. D. (2023). Conr: Contrastive regularizer for deep imbalanced regression. arXiv preprint arXiv:2309.06651
2023 arXiv
-
[21]
Krawczyk, B. (2016). Learning from imbalanced data: open challenges and future directions. Progress in Artificial Intelligence , 5(4):221--232
2016
-
[22]
and Poirier, S
LeDell, E. and Poirier, S. (2020). H2O A uto ML : Scalable automatic machine learning. 7th ICML Workshop on Automated Machine Learning (AutoML)
2020
-
[23]
Noisy replication in skewed binary classification
Lee and Sauchi (2000). Noisy replication in skewed binary classification. Computational Statistics and Data Analysis , 34(2):165--191
2000
-
[24]
Lin, T.-Y., Goyal, P., Girshick, R., He, K., and Doll \'a r, P. (2017). Focal loss for dense object detection. In Proceedings of the IEEE international conference on computer vision , pages 2980--2988
2017
-
[25]
M., and Zhang, C
Ma, C., Tschiatschek, S., Turner, R., Hern \'a ndez-Lobato, J. M., and Zhang, C. (2020). Vaem: a deep generative model for heterogeneous mixed type data. Advances in Neural Information Processing Systems , 33:11237--11247
2020
-
[26]
Training and assessing classification rules with imbalanced data
Menardi and Torelli (2014). Training and assessing classification rules with imbalanced data. Data Mining and Knowledge Discovery , 28(1):92--122
2014
-
[27]
Moniz, N., Ribeiro, R., Cerqueira, V., and Chawla, N. (2018). Smoteboost for regression: Improving the prediction of extreme values. In 2018 IEEE 5th international conference on data science and advanced analytics (DSAA) , pages 150--159. IEEE
2018
-
[28]
Patki, N., Wedge, R., and Veeramachaneni, K. (2016). The synthetic data vault. In IEEE International Conference on Data Science and Advanced Analytics (DSAA) , pages 399--410
2016
-
[29]
Ren, J., Zhang, M., Yu, C., and Liu, Z. (2022). Balanced mse for imbalanced visual regression. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 7926--7935
2022
-
[30]
Ribeiro, R. P. (2011). Utility-based regression. Ph. D. dissertation
2011
-
[31]
Scott, D. W. (2015). Multivariate density estimation: theory, practice, and visualization . John Wiley & Sons
2015
-
[32]
P., and Chakraborty, P
Sen, S., Singh, K. P., and Chakraborty, P. (2023). Dealing with imbalanced regression problem for large dataset using scalable artificial neural network. New Astronomy , 99:101959
2023
-
[33]
and Armon, A
Shwartz-Ziv, R. and Armon, A. (2022). Tabular data: Deep learning is not all you need. Information Fusion , 81:84--90
2022
-
[34]
and Young, G
Silverman, B. and Young, G. (1987). The bootstrap: to smooth or not to smooth? Biometrika , 74(3):469--479
1987
-
[35]
Silverman, B. W. (1986). Density estimation for statistics and data analysis , volume 26. CRC press
1986
-
[36]
Y., Dao, N., and Branco, P
Song, X. Y., Dao, N., and Branco, P. (2022). Distsmogn: Distributed smogn for imbalanced regression problems. In Fourth International Workshop on Learning with Imbalanced Domains: Theory and Applications , pages 38--52. PMLR
2022
-
[37]
Steininger, M., Kobs, K., Davidson, P., Krause, A., and Hotho, A. (2021). Density-based weighting for imbalanced regression. Machine Learning , 110:2187--2211
2021
-
[38]
Stocksieker, S., Pommeret, D., and Charpentier, A. (2023). Data augmentation for imbalanced regression. In International Conference on Artificial Intelligence and Statistics , pages 7774--7799. PMLR
2023
-
[39]
P., Pfahringer, B., and Branco, P
Torgo, L., Ribeiro, R. P., Pfahringer, B., and Branco, P. (2013). Smote for regression. In Portuguese conference on artificial intelligence , pages 378--389. Springer
2013
-
[40]
and Partin-Vaisband, I
Utyamishev, D. and Partin-Vaisband, I. (2019). Progressive vae training on highly sparse and imbalanced data. arXiv preprint arXiv:1912.08283
2019 arXiv
-
[41]
and Wang, H
Wang, Z. and Wang, H. (2024). Variational imbalanced regression: Fair uncertainty quantification via probabilistic smoothing. Advances in Neural Information Processing Systems , 36
2024
-
[42]
Wu, W., Kunz, N., and Branco, P. (2022). Imbalancedlearningregression-a python package to tackle the imbalanced regression problem. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases , pages 645--648. Springer
2022
-
[43]
Xu, L., Skoularidou, M., Cuesta-Infante, A., and Veeramachaneni, K. (2019). Modeling tabular data using conditional gan. In Advances in Neural Information Processing Systems
2019
-
[44]
Yang, Y., Zha, K., Chen, Y., Wang, H., and Katabi, D. (2021). Delving into deep imbalanced regression. In International Conference on Machine Learning , pages 11842--11851. PMLR
2021
-
[45]
Zhang, C., Zhou, Y., Chen, Y., Deng, Y., Wang, X., Dong, L., and Wei, H. (2018). Over-sampling algorithm based on vae in imbalanced classification. In Cloud Computing--CLOUD 2018: 11th International Conference, Held as Part of the Services Conference Federation, SCF 2018, Seat...
2018
Reviewed August 11, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.