{"id":"d67f6614-174f-4486-ba9e-96ff4e135a54","arxiv_id":"2505.16074","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":3,"one_line_summary":"A single shared-weight network that encodes on the forward pass and decodes on the backward pass can halve VAE parameters while keeping performance roughly on par with a standard VAE.","lead":"The paper proposes a variational autoencoder that uses one shared neural network for both encoding and decoding, running forward to encode and backward to decode. The design cuts the parameter count almost in half and roughly matches or slightly beats a standard VAE on MNIST, Fashion-MNIST, CIFAR-10, and CelebA.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"BELBO is a standard ELBO with tied weights; the load-bearing risk is that the 'slightly outperformed' claim rests on single runs and mixed tables, not on the bound.","rationale":"I read the reader's weakest assumption as the claimed invalidity of the BELBO bound. On close inspection, the bound in Section III is mathematically valid if the model's decoder is taken to be pb(x|z,θ): the inequality ln p_model(x) ≥ E_qf[ln pb(x|z,θ)] - KL(qf||p(z)) is the textbook ELBO for the generative model p_model(x)=∫pb(x|z,θ)p(z)dz. The paper's loose notation (calling p(x|z,θ) the decoder in the derivation and pb the decoder in the algorithm) creates an apparent mismatch, but no separate 'true conditional likelihood' exists in variational inference; the decoder defines the model. So the theoretical objection is largely terminological and repairable. What actually threatens the central claim is the empirical basis for 'slightly outperformed': single runs, no error bars, no code, and tables that show bidirectional variants losing on several metrics. The parameter-count advantage is structurally guaranteed, but the performance advantage is not. A multi-seed rerun with paired statistics would settle whether the observed differences are real or noise. This reinforces the reader's CONDITIONAL verdict rather than changing it, so I recommend UNCHANGED.","tokens_in":12670,"tokens_out":9596,"duration_ms":83783,"concrete_test":"Rerun every model-pair in Tables I-IV with at least 5 random seeds each, report mean and standard deviation, and apply a paired Wilcoxon signed-rank test over all metrics; if the bidirectional variant does not win a majority of paired comparisons at p<0.05, the claim of 'slightly outperformed' should be withdrawn.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that BVAEs 'slightly outperformed' unidirectional VAEs is not established by the reported experiments. Each configuration appears to be run once: no random seeds, no error bars, and no paired significance tests are given. The tables are internally mixed. For example, on CIFAR-10 the vanilla BVAE has better NLL (1814.1 vs 1817.0) but worse downstream accuracy (50.55% vs 51.73%); on Fashion-MNIST the bidirectional IWAE has worse FID (2.881 vs 2.489) and worse accuracy (88.05% vs 88.14%); on CelebA the bidirectional β-VAE has substantially worse NLL (6261.9 vs 6243.0). With differences this small and inconsistent across tasks, the abstract's 'slightly outperformed' may reflect initialization noise rather than a real effect. The BELBO derivation itself is repairable: if pb(x|z,θ) is taken to define the generative likelihood, Eq (38) is exactly the standard ELBO for p_model(x)=∫pb(x|z)p(z)dz, so the apparent 'mismatch' in Section III is mostly a notation problem. Thus the load-bearing weak point is empirical reproducibility, not the lower-bound algebra.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a bidirectional variational autoencoder (BVAE) in which a single network is used both as the encoder, through a forward pass with likelihood q_f(z|x,θ), and as the decoder, through a backward pass with likelihood p_b(x|z,θ). The authors derive a lower bound they call BELBO, which parallels the standard ELBO, and report experiments on MNIST, Fashion-MNIST, CIFAR-10, and CelebA-64 comparing vanilla VAEs, β-VAEs, β-TCVAEs, and IWAEs with their bidirectional counterparts across reconstruction, classification, interpolation, and generation. The central claims are that the bidirectional architecture approximately halves the parameter count and that it still slightly outperforms unidirectional VAEs.","tokens_in":12877,"tokens_out":5625,"duration_ms":51578,"significance":"If the empirical claims were established, the paper would offer a simple and parameter-efficient VAE variant with tied encoder/decoder weights, which is of practical interest. The paper deserves credit for presenting a concrete architecture and for a mostly correct review of the standard ELBO derivation in Section II. However, the theoretical contribution is currently overstated: once the backward-pass density p_b is properly declared as the generative decoder, BELBO is the standard ELBO for the tied-weight model, not a new lower bound on the true data likelihood. The headline empirical claim is also not supported by the tables as reported, because each configuration appears to be a single run with no error bars or significance tests and the results are inconsistent across metrics and model families. The paper does not provide code, random seeds, or detailed training configurations, which further limits reproducibility.","major_comments":[{"comment":"The BELBO derivation is not yet a proof that the training objective bounds the data log-likelihood. Eqs. (26)–(38) are written with p(x|z,θ) as the decoder, but Algorithm 1 and the surrounding text replace this density with the backward-pass likelihood p_b(x|z,θ). The manuscript never defines the generative model p_model(x)=∫ p_b(x|z,θ)p(z|θ)dz, nor does it state that the inequality in Eq. (37) is intended for that model. If p_b is taken as the decoder likelihood, Eq. (38) is simply the standard ELBO for a tied-weight model, and the derivation is complete but not novel; if p_b is not the true conditional likelihood, then no correction term is provided and the bound on ln p(x|θ) is unproven. Please state the model explicitly and adjust the claim accordingly.","section":"III, Eqs. (26)–(38) and Algorithm 1, Step 8"},{"comment":"The abstract's claim that BVAEs 'slightly outperformed' unidirectional VAEs is not established by the reported experiments. No random seeds, repeated runs, error bars, or paired significance tests are given, so the observed differences may be initialization noise. The results are also internally inconsistent across metrics: in Table I the vanilla BVAE has better NLL (86.69 vs 86.72) but worse FID (3.370 vs 3.340), and the bidirectional IWAE has better NLL (86.00 vs 86.02) but worse FID (3.330 vs 2.958); in Table II the bidirectional IWAE has worse FID (2.881 vs 2.489) and worse accuracy (88.05% vs 88.14%); in Table IV the bidirectional β-VAE has substantially worse NLL (6261.9 vs 6243.0). Either provide multiple seeds with standard deviations and significance tests, or scale back the headline claim to one of parameter-count reduction with comparable performance on selected metrics.","section":"Tables I–IV and Abstract"},{"comment":"The relationship between the bidirectional backpropagation objective and the BELBO used for training is not specified. Eq. (2) defines B-BP as maximizing q_f(z|x,θ)p_b(x|z,θ), equivalently maximizing ln q_f(z|x,θ)+ln p_b(x|z,θ). Algorithm 1 maximizes BCE+KLD, i.e., E_{q_f}[ln p_b(x|z,θ)] - KL(q_f||p(z)), which is a different functional: the former contains no expectation over the variational distribution and no prior KL term. The text cites [2] for B-BP but does not show that B-BP implements BELBO maximization. Please clarify how the update in Eq. (39) is derived from the B-BP algorithm, or present Algorithm 1 as the definition of BVAE training and describe its relation to Eq. (2).","section":"I, Eqs. (2)–(3), and III, Algorithm 1"}],"minor_comments":[{"comment":"The estimated objective is written as \\tilde{L}_{ELBO}(x,θ), but in the BVAE setting it should be \\tilde{L}_{BELBO}(x,θ) to match the notation of Section III.","section":"Algorithm 1, Step 8"},{"comment":"The sentence 'the forward pass p_f(x|z,θ) to approximate the encoding model' should read 'decoding model'; p_f(x|z,θ) is a decoder likelihood, not an encoder.","section":"II, end of Section II"},{"comment":"The sentence 'because the negative of the natural logarithm is convex' is imprecise; the argument applies Jensen's inequality to the concave logarithm and then uses the nonnegativity of the negative of that inequality. Please rephrase.","section":"II, Eqs. (16)–(21)"},{"comment":"Several numerical entries are difficult to read because of spacing artifacts, e.g., '18 4 .340' in Table I and '2 .483' in Table III. Please reformat the tables so that every number is unambiguous.","section":"Tables I–IV"},{"comment":"References [6] and [7] are the same arXiv paper and ICLR paper by Burda, Grosse, and Salakhutdinov; they should be consolidated into a single citation.","section":"References"},{"comment":"The active-units metric should be defined as the variance over the data of the posterior mean, Var_x(E_{z|x}[z_d]), not 'Cov_x' as written, and the threshold should be stated precisely.","section":"IV-E, Eq. (41)"}],"recommendation":"major_revision","confidential_remarks":"The paper relies heavily on the authors' prior bidirectional backpropagation work [1,2], and the novelty relative to tied-weight autoencoders should be clarified in revision. The most serious issue is that the experimental section, as currently written, does not support the 'slightly outperformed' claim in the abstract; if the authors cannot provide multi-seed results, they should reframe the contribution as a parameter-halving variant with comparable performance. The theoretical issue in Section III is repairable by defining the generative model explicitly, but until that is done the paper overstates the status of BELBO as a bound on the true data log-likelihood."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a tied-weight VAE, not a new objective. The BELBO is the standard ELBO with a shared theta, and the derivation is repairable if you define the generative likelihood as pb(x|z,θ). The genuinely interesting bit is empirical: one network running forward for encode and backward for decode cuts parameters in half across four VAE variants and four datasets. That is worth knowing, but the headline claim that BVAEs 'slightly outperformed' is not backed by the tables.\n\nWhat's new: the specific architecture — same weight matrices for convolution and transposed convolution — and the breadth of the comparison. They test vanilla VAE, beta-VAE, beta-TCVAE, and IWAE on MNIST, Fashion-MNIST, CIFAR-10, and CelebA-64, reporting NLL, FID, PSNR, SSIM, and classification accuracy. The parameter savings are real and consistent: each BVAE uses roughly half the memory of its unidirectional counterpart. They also report cases where BVAE loses (e.g., CelebA beta-VAE NLL is 6261.9 vs 6243.0), so they are not hiding the messy picture.\n\nSoft spots: no random seeds, no error bars, no significance tests. The differences are often tiny — a few hundredths of a nat or a point of accuracy — and the direction flips depending on dataset and metric. With single runs, the 'slightly outperformed' abstract claim is just not established. The theory section also never explicitly states that the generative model is defined by pb; it just substitutes pb into equations that still say p(x|z,θ). A referee can fix this, but it is a real presentation flaw.\n\nBottom line: the idea has some merit for memory-limited deployment, and the experiments cover enough ground to be useful, but the evidence needs to be re-run with seeds and error bars. A serious referee could get this into acceptable shape, so I would send it to peer review with major revisions.","headline":"A tied-weight VAE with a half-parameter architecture and mixed empirical evidence; the theory is standard ELBO, the experiments need seeds and error bars.","tokens_in":13459,"tokens_out":3868,"would_cite":false,"duration_ms":31256,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A single network can do both encoding and decoding in a variational autoencoder, cutting parameters nearly in half without losing performance.","keywords":["bidirectional variational autoencoder","variational autoencoder","evidence lower bound","bidirectional backpropagation","directional likelihoods","image generation","weight tying","representation learning"],"falsifier":"Compute an unbiased Monte Carlo estimate of the true held-out log-likelihood for a trained BVAE on a standard dataset and check whether the BELBO value reported by the same model exceeds that estimate. If the BELBO is larger than the true log-likelihood on any sample, the claimed bound fails. A second check is to train a tied-weight autoencoder with the same architecture but no variational objective; if it matches the BVAE on all tasks, the variational machinery adds nothing beyond weight tying.","tokens_in":12411,"feed_emoji":"🔄","tokens_out":7402,"duration_ms":53054,"temperature":0.7,"pith_summary":"The paper proposes the bidirectional variational autoencoder (BVAE), which replaces the usual two-network encoder-decoder setup of a variational autoencoder with one network that encodes on its forward pass and decodes on its backward pass through the same synaptic weights. Training maximizes the BELBO, a bidirectional version of the evidence lower bound, using a bidirectional backpropagation algorithm that jointly optimizes the forward and backward directional likelihoods. The paper reports that on MNIST, Fashion-MNIST, CIFAR-10, and CelebA-64, BVAEs use almost 50% fewer parameters than comparable unidirectional VAEs and still perform slightly better on reconstruction, classification, interpolation, and generation. If the central claim holds, VAE training and deployment would cost roughly half as much in memory and parameter storage for equal or better quality.","feed_headline":"Bidirectional VAE halves parameters, slightly beats standard VAE","feed_subtitle":"One network encodes forward and decodes backward through the same weights, nearly halving VAE cost.","key_machinery":"The central mechanism is the BELBO (bidirectional evidence lower bound), built on the directional likelihoods of bidirectional backpropagation: the forward likelihood $q_f(z|x,\\theta)$ encodes the latent variable and the backward likelihood $p_b(x|z,\\theta)$ decodes it through the same synaptic web. The bound is obtained by inserting these two densities into the identity for the data log-likelihood and discarding the nonnegative KL divergence between $q_f$ and the true posterior, which turns the intractable marginal likelihood into a tractable one-parameter objective. The BVAE training loop estimates this bound with the reparameterization trick for the latent sample and updates the shared weight vector with ordinary backpropagation through the BELBO estimate.","core_discovery":"The central claim is that the encoder and decoder of a variational autoencoder need not be separate networks: a single parameterized network can serve both roles if it is trained with bidirectional backpropagation. In a BVAE, the forward likelihood $q_f(z|x,\\theta)$ approximates the intractable posterior $q(z|x,\\theta)$ and the backward likelihood $p_b(x|z,\\theta)$ approximates the decoder likelihood $p(x|z,\\theta)$, with both densities sharing the same weight vector $\\theta$. The paper derives the BELBO by substituting these directional likelihoods into the standard log-likelihood identity and dropping a nonnegative KL divergence term, giving a one-parameter objective that the network maximizes. The simulations compare vanilla VAEs, $\\beta$-VAEs, $\\beta$-TCVAEs, and IWAEs with their bidirectional counterparts and report that the bidirectional versions halve the parameter count while slightly outperforming the unidirectional ones on the tested image tasks.","pith_inferences":["A direct test of the variational claim would be to compare the BELBO value reported during training against an unbiased Monte Carlo estimate of the true held-out log-likelihood; if the BELBO ever exceeds that estimate, the bound is not a true lower bound and the method's value rests on the empirical results alone.","The backward-pass density $p_b(x|z,\\theta)$ is the transpose-weight mapping, not a freely parameterized decoder; datasets with strongly nonlinear conditional structure might show a larger performance gap because the decoder's capacity is constrained by the encoder's weights.","The same bidirectional weight-sharing idea could be applied to other latent-variable generative models, such as hierarchical VAEs or normalizing flows, whenever a backward pass can approximate the inverse of the forward map."],"forward_implications":["If the central claim is correct, a variational autoencoder can be trained and run with a single network, cutting parameter storage and memory roughly in half while maintaining or slightly improving image reconstruction, classification, generation, and interpolation performance.","The same shared-weight substitution applies to VAE variants such as $\\beta$-VAE, $\\beta$-TCVAE, and IWAE, so the parameter savings should carry over to any objective built on the ELBO.","The backward pass uses transposed convolution with the same masks as the forward convolution, so the architecture enforces a form of weight tying that acts as a regularizer and may explain the small generalization gains reported in the simulations.","Because the savings scale with the number of trainable weights, deeper and wider autoencoders would benefit more from the bidirectional architecture than the relatively shallow networks tested here."],"supporting_citations":[{"why":"Supplies the variational autoencoder objective and the ELBO that the BVAE adapts into the BELBO.","marker":"[19]"},{"why":"Introduces the bidirectional backpropagation algorithm that provides the forward and backward directional likelihoods.","marker":"[2]"},{"why":"Earlier work on bidirectional representation learning that underlies the shared-weight architecture.","marker":"[1]"},{"why":"Defines the $\\beta$-VAE baseline whose bidirectional version is implemented and compared.","marker":"[17]"},{"why":"Defines the IWAE baseline whose bidirectional version is implemented and compared.","marker":"[6]"},{"why":"Defines the $\\beta$-TCVAE baseline whose bidirectional version is implemented and compared.","marker":"[11]"},{"why":"Presents bidirectional autoencoders, the prior architecture that BVAE extends with joint likelihood optimization.","marker":"[5]"}],"fun_headline_variants":["One network, two directions: VAE with half the parameters","Single network does both: VAE parameters nearly halved","Half the parameters, better results: the bidirectional VAE","Same weights, both directions: VAE with half the parameters","Bidirectional VAE: one network, half parameters, better results"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole variational argument depends on the backward-pass density $p_b(x|z,\\theta)$ actually equaling the true decoder likelihood $p(x|z,\\theta)$ for the data distribution; if it does not, the BELBO is not a lower bound on the log-likelihood and the architecture is just a tied-weight autoencoder whose performance rests on the simulations.","fun_headline_variants_meta":{"raw":{"variants":["One network, two directions: VAE with half the parameters","Single network does both: VAE parameters nearly halved","Half the parameters, better results: the bidirectional VAE","Same weights, both directions: VAE with half the parameters","Bidirectional VAE: one network, half parameters, better results"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00061,"raw_usage":{"total_tokens":2788,"prompt_tokens":841,"completion_tokens":1947,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":457,"completion_tokens_details":{"reasoning_tokens":1862}},"tokens_in":457,"tokens_out":1947,"duration_ms":12360,"temperature":1.0,"reasoning_tokens":1862,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T15:07:51.027119+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compute an unbiased Monte Carlo estimate of the true held-out log-likelihood for a trained BVAE on a standard dataset and check whether the BELBO value reported by the same model exceeds that estimate. If the BELBO is larger than the true log-likelihood on any sample, the claimed bound fails. A second check is to train a tied-weight autoencoder with the same architecture but no variational objective; if it matches the BVAE on all tasks, the variational machinery adds nothing beyond weight tying.","supporting_citations":[{"cited_title":"Bidirectional backpropagation,","cited_arxiv_id":null,"evidence_quote":"Introduces the bidirectional backpropagation algorithm that provides the forward and backward directional likelihoods."},{"cited_title":"Bidirectional representation and backpropa- gation learning,","cited_arxiv_id":null,"evidence_quote":"Earlier work on bidirectional representation learning that underlies the shared-weight architecture."},{"cited_title":"beta-vae: Learning basic visual concepts with a constrained variational framework,","cited_arxiv_id":null,"evidence_quote":"Defines the $\\beta$-VAE baseline whose bidirectional version is implemented and compared."},{"cited_title":"Bidirectional backpropagation autoencoding networks for image compression and denoising,","cited_arxiv_id":null,"evidence_quote":"Presents bidirectional autoencoders, the prior architecture that BVAE extends with joint likelihood optimization."}],"review_version":1}