{"id":"d0693216-3741-4c5c-8d1c-df20e7a3074a","arxiv_id":"2507.04385","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"Autoencoding probabilistic circuits train a single probabilistic circuit to jointly model data and explicit embedding variables, enabling end-to-end autoencoding with neural decoders and robust encoding under missing data.","lead":"This paper introduces autoencoding probabilistic circuits (APCs), which use a probabilistic circuit to model the joint distribution of data and embeddings and then encode inputs by sampling the embedding given observed data. The approach natively handles missing data, reconstructs images and tabular data better than several neural and circuit autoencoder baselines, and supports data-free knowledge distillation from pretrained VAEs.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central claim depends on an unprincipled random coupling of embedding leaves to data leaves; if pC(Z|X=x) is uninformative under reasonable placement variants, the reported gains may be architecture-specific rather than a property of the tractable encoder framework.","rationale":"The paper makes a concrete, falsifiable claim: a PC encoding pC(X,Z) with embeddings sampled from pC(Z|X=x) yields better reconstruction and missing-data robustness than existing PC-based and neural autoencoders. The mechanism is sound in principle: smoothness and decomposability give exact marginals and conditional sampling, and the appendix's wake-phase argument shows the L_NLL term is a valid stochastic gradient of the marginal log-likelihood. The strongest part of the evidence is the consistent pattern across eight image datasets and twenty tabular datasets, plus ablations showing each proposed component contributes. However, the encoder's informativeness is not guaranteed by the framework; it is an empirical property of the specific random leaf-coupling heuristic. The paper itself acknowledges this is a heuristic and leaves principled placement to future work. This is exactly the reader's weakest assumption, and I agree it is the most load-bearing open issue. It does not invalidate the paper's empirical claims, which are extensive and internally consistent, but it does mean the headline result may not generalize to other circuit topologies or embedding-placement strategies. A single controlled placement experiment would settle whether the reported gains are intrinsic to the tractable-encoder idea or contingent on the particular heuristic. Since the reader already reached CONDITIONAL based on this and supporting concerns (missing code, zero standard deviations), my read does not move the verdict.","tokens_in":35472,"tokens_out":8506,"duration_ms":107140,"concrete_test":"Retrain APC on MNIST and CIFAR with three embedding placements: (1) the paper's random single-pixel coupling, (2) each embedding coupled to a random subset of 10% of data leaves at the input layer, and (3) embeddings attached only at the top sum layer just below the root. For each variant, measure reconstruction MSE and downstream linear-probe accuracy at 0%, 50%, and 90% MCAR, and also measure the reconstruction gap between z sampled from pC(Z|X=x) and z sampled from the prior pC(Z). If placement (1) is not among the best variants, or if the conditional-vs-prior reconstruction gap is smaller than a 20% MSE reduction, the random coupling is a load-bearing heuristic and the central claim needs qualification.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The framework's encoder is defined by a heuristic: embedding random variables are inserted as leaf input units and, for image data, each embedding is randomly coupled with a single data variable unit via a local product unit (Section 3.1, 'Encoder Structure'; Appendix B.1). The paper explicitly defers principled placement to future work ('the circuit structures employed in this work were selected heuristically'). All headline results—lower reconstruction MSE under MCAR/MAR corruption, stable downstream accuracy, and robust knowledge distillation—flow through this specific architecture. Yet no experiment varies the coupling strength, embedding depth, or pairing rule, and no measurement establishes that the learned conditional pC(Z|X=x) carries enough information about X for the neural decoder to exploit. If a reasonable placement (e.g., embeddings attached at a higher layer, or coupled to many data variables) yields a posterior that is nearly independent of X, then the decoder receives uninformative samples and the missing-data robustness claims collapse. The ablation in Table 1 shows each loss/decoder component contributes, but it does not test the embedding-placement design, which is the least principled and most load-bearing component of the framework.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces autoencoding probabilistic circuits (APCs), a framework in which a smooth and decomposable probabilistic circuit explicitly models the joint distribution pC(X, Z) over data and embedding variables. Encoding is performed by tractable conditional sampling z ~ pC(Z | X = x), which natively handles missing inputs through exact marginalization, and decoding is performed by a neural network trained end-to-end with a weighted combination of reconstruction, KLD-prior, and joint-likelihood losses, using SIMPLE-based differentiable sampling. The empirical sections compare APCs with SPAE, VAE, MIWAE, and missForest on eight image datasets and twenty DEBD tabular datasets, evaluate downstream linear classification and embedding visualizations, report ablations of each loss and decoder component, and explore data-free knowledge distillation and embedding-based OOD likelihood separation.","tokens_in":35653,"tokens_out":4881,"duration_ms":58997,"significance":"If the reported results hold, APCs provide a genuinely different way to build autoencoders: the encoder is a tractable probabilistic model rather than a neural network, which yields exact conditional inference and intrinsic handling of missing data without imputation. The paper's strengths are its broad empirical coverage (eight image and twenty tabular datasets), the use of the same neural decoder across APC and VAE comparisons, consistent ablations in Table 1, and the improvement of differentiable PC sampling via SIMPLE, which is validated in a controlled experiment. The main weakness is that the core embedding-placement mechanism is heuristic and is never varied or measured, and the code is not actually available, so the generality of the central claim remains partially open.","major_comments":[{"comment":"The central mechanism of the framework is the placement of embedding variables as leaf units randomly paired with data input units, but the manuscript contains no experiment that varies this design choice. Since the authors explicitly state that 'the circuit structures employed in this work were selected heuristically' (Section 6), the headline claim that a tractable PC encoder is responsible for the observed missing-data robustness is not yet separated from the specific random-coupling heuristic. I ask for ablations that vary (i) the number of data units each embedding is coupled with (e.g., 1 vs. k vs. all), (ii) the depth at which embeddings are inserted, and (iii) random vs. structured pairing, measuring reconstruction MSE and downstream accuracy under MCAR corruption; this would establish whether the reported advantages are a property of the framework or of one architecture.","section":"§3.1, Appendix B.1"},{"comment":"The vanilla VAE baseline is evaluated with constant zero imputation, and although the text states that mean imputation was also explored, no quantitative results are reported for it. Because zero imputation is known to be a particularly weak treatment of missing inputs, the comparisons in Figs. 5-7 and Tables 2-3 may overstate the gap between APCs and neural autoencoders. The authors should report the mean-imputation VAE results, add a masked or missing-data-specific neural baseline such as mDAE or not-MIWAE, and clarify whether reconstruction MSE is computed only on missing entries or on the full image, since this metric choice directly affects the interpretation of the robustness curves.","section":"Appendix B, 'Missing Data'"},{"comment":"The implementation link is given as 'https://github.com/placeholder-url', so no runnable code or complete configuration files are available to the reader. Given that the empirical evaluation is the main support for the paper's claims and that several architecture details are delegated to source files, this placeholder makes the results impossible to verify. Please provide a working code release or, failing that, full hyperparameter tables, circuit construction parameters (including the embedding-pairing rule), and dataset preprocessing details in the appendix.","section":"Appendix B"}],"minor_comments":[{"comment":"The word 'mechansim' should be 'mechanism'.","section":"§3.4"},{"comment":"The caption contains a typo: 'linearly seperable' should be 'linearly separable'.","section":"Figure 7 caption"},{"comment":"The text '100,0000 iterations' appears to be a typo and should read '100,000 iterations'.","section":"Appendix B, 'Downstream Task Training'"},{"comment":"The reconstruction loss is written as a negative log-likelihood, but the text says MSE is used as L_REC; please clarify the connection (e.g., a Gaussian observation model) and define how the reconstruction \\hat{x}_i is computed from the sampled embedding z_i.","section":"Eq. (2)"},{"comment":"In the pseudocode, the conditions 'if n == ' and 'if n == ⊗' appear to have missing symbols due to formatting; please restore the unit-type predicates so the algorithm is readable.","section":"Appendix Algorithm 1"}],"recommendation":"major_revision","confidential_remarks":"The paper is within scope for cs.LG and the central formulation is sound and original. My main concerns are the untested sensitivity to the heuristic embedding-placement design and the absence of runnable code, both of which are addressable within the manuscript's scope; I therefore recommend major revision rather than rejection. I did not find citation or novelty-disclosure problems."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe short version: this is a genuine contribution, not a repackaging. Braun et al. introduce autoencoding probabilistic circuits (APCs), where embeddings are explicit random variables in a smooth and decomposable PC, and the encoder is the exact conditional pC(Z|X). They train the whole pipeline end-to-end with a neural decoder using SIMPLE, a gradient estimator that handles the discrete choices in the circuit. The empirical work is broad: eight image datasets, 20 tabular DEBD datasets, MCAR and MAR corruption, comparisons against SPAE, VAE, MIWAE, and missForest, plus ablations of each component. Their central claim—APCs stay usable at very high missing-data rates where neural autoencoders collapse—holds up in the reported numbers, and the downstream classification results are consistent with it.\n\nWhat is genuinely new: SPAE extracts embeddings post-hoc from an MLE-trained circuit; APCs model the joint distribution over data and embeddings and train the circuit against reconstruction. That is a structural difference, not a tweak. The knowledge-distillation application is a nice bonus, showing the tractable encoder can transfer from a VAE teacher without the original data.\n\nSoft spots, in proportion. (1) No code: the GitHub URL in the paper is a literal placeholder. For a paper whose empirical claims are the main selling point, that is the biggest issue. (2) The embedding placement heuristic—randomly coupling embedding leaves to data leaves—is acknowledged as a design choice but never stress-tested. The worry that pC(Z|X) might be uninformative under different placements is plausible in principle, but the consistent success across many datasets suggests the posterior is informative enough. A sensitivity analysis varying coupling depth or the number of coupled variables would materially strengthen the paper. (3) The L_NLL term trains the model to assign high likelihood to its own samples. It is self-referential, but the ablation shows it helps, and it does not invalidate comparisons to external baselines. (4) Some reported standard deviations are exactly 0.00; probably rounding of small variance, but worth checking.\n\nThe vanilla VAE baseline uses zero imputation, which is weak, but the paper also beats MIWAE, purpose-built for missing data, so the main comparison is sound.\n\nWho this is for: anyone working on tractable probabilistic models, representation learning, or missing-data robustness. It deserves a serious referee—the idea is novel, the experiments are broad, and the flaws are fixable rather than fundamental. I would recommend engaging and asking for code release and a placement sensitivity analysis in revision.","headline":"A genuinely new way to make probabilistic circuits learn representations, with strong missing-data robustness claims that mostly hold up; the missing code and unexamined embedding-placement heuristic are the main points to press.","tokens_in":36256,"tokens_out":4109,"would_cite":true,"duration_ms":43231,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Modeling data and embeddings inside one probabilistic circuit makes autoencoders robust to missing data.","keywords":["probabilistic circuits","autoencoding","representation learning","missing data","differentiable sampling","tractable inference","variational autoencoders","knowledge distillation"],"falsifier":"Train an APC on an image dataset with deliberately adversarial pairings, such as pairing every embedding variable with a fixed pixel or with pixels that carry no class information, and re-run the reconstruction and downstream experiments; if performance collapses to near the VAE baseline, the robustness claim depends on the random-coupling heuristic rather than on the tractable-conditional mechanism itself. Alternatively, measure the conditional mutual information $I(X;Z)$ under the learned circuit; if it is close to zero at high corruption levels while reconstruction still succeeds, the information must flow through the decoder rather than the encoder, which would contradict the paper's attribution.","tokens_in":35235,"feed_emoji":"🧩","tokens_out":4798,"duration_ms":44500,"temperature":0.7,"pith_summary":"This paper introduces autoencoding probabilistic circuits (APCs), a way to do representation learning in which the probabilistic encoder is itself a probabilistic circuit rather than a neural network. The proposal is to model the joint distribution $p_{\\mathcal{C}}(X,Z)$ of data $X$ and embedding variables $Z$ inside one smooth and decomposable circuit, and to encode an input by sampling $z\\sim p_{\\mathcal{C}}(Z\\mid X=x)$ or by most probable explanation inference. Because circuits support tractable marginalization, the same encoder answers the encoding query when parts of $x$ are missing, by integrating them out instead of imputing them. The paper claims this yields reconstructions that beat existing circuit-based autoencoders and match or beat neural autoencoders once data is missing, with embeddings that stay linearly separable at high corruption levels. The reason to care is that a tractable, probabilistic encoder with an explicit posterior makes missing-data handling a native property rather than a heuristic.","feed_headline":"Probabilistic circuits make autoencoders robust to missing data","feed_subtitle":"A single circuit models data and embeddings, so partial inputs can be encoded without imputation.","key_machinery":"The load-bearing object is a smooth and decomposable probabilistic circuit $\\mathcal{C}$ that contains embedding random variables $Z$ as first-class input units alongside data inputs $X$, with Gaussian input units for embeddings. Smoothness and decomposability guarantee that marginals, conditionals, and samples can be computed exactly in time linear in the circuit size; encoding is a two-pass procedure that caches per-input likelihoods and reweights sum-unit weights, then samples a path using SIMPLE, a $k$-subset gradient estimator (with $k=1$) that keeps the forward pass discrete and the backward pass differentiable. This differentiable sampling is what lets the neural decoder's gradients reach the circuit encoder. The neural decoder is the second half of the machinery: it maps sampled embeddings back to data, and because the encoder always returns a complete embedding even from partial evidence, the decoder never sees missing entries.","core_discovery":"The central claim is that explicit random variables for embeddings, placed as input units inside a smooth and decomposable probabilistic circuit, turn the circuit into an autoencoder whose encoder is an exact conditional distribution. Concretely, APCs build a single PC over data and embeddings, obtain embeddings by tractable conditional sampling $z\\sim p_{\\mathcal{C}}(Z\\mid X_o=x_o)$ for any observed subset $X_o$, and train end to end against a neural decoder using a three-term loss: reconstruction log-likelihood, KL divergence of the sampled embedding distribution to a prior, and joint data-embedding negative log-likelihood. The paper reports that APCs outperform SPAE, VAE, MIWAE, and missForest in reconstruction under MCAR and MAR corruptions across eight image datasets and 18 of 20 tabular datasets, keep downstream classification accuracy high under missingness, and can distill a pretrained VAE's knowledge without the original data. In the authors' framing, the encoder is the joint distribution itself, so no separate inference network is needed.","pith_inferences":["The random local coupling of each embedding unit to one data unit is the weakest design choice; a principled placement rule, such as learned or information-theoretic coupling, could either strengthen the claim or reveal its limits.","The framework's missing-data claim should transfer to structured corruptions beyond images, but the MAR experiments cover only geometric patterns; heterogeneous tabular missingness is a natural stress test.","Tractable embedding likelihoods suggest a direct test of representation quality: measuring how well $p_{\\mathcal{C}}(Z)$ separates classes or domains, rather than relying only on reconstruction or downstream accuracy.","Because the encoder is a circuit, the same machinery could support probabilistic retrieval or active querying, where the model chooses which variables to observe next based on exact conditional information."],"forward_implications":["Autoencoders can be built with a tractable probabilistic encoder, so encoding queries like $p(Z\\mid X_o=x_o)$ are exact rather than approximate.","Missing data at test time requires no imputation: the marginalization at the input units is a native operation of the circuit.","End-to-end training with a neural decoder is possible through differentiable sampling, and the ablation identifies all three losses plus the neural decoder as necessary for the reported performance.","A pretrained VAE can be distilled into an APC without the training data, transferring the teacher's distribution while gaining missing-data robustness.","Embedding likelihoods $\\log p_{\\mathcal{C}}(Z)$ are tractable, giving a principled score for out-of-distribution detection."],"supporting_citations":[{"why":"Defines the prior PC-based autoencoding scheme (CAT and ACT embeddings) that APCs compare against and extend.","marker":"Vergari et al. (2018)"},{"why":"Supplies the differentiable sampling procedure for PCs that APCs adopt and improve with SIMPLE.","marker":"Lang et al. (2022)"},{"why":"Provides the SIMPLE gradient estimator for k-subset sampling used in the circuit's sum units.","marker":"Ahmed et al. (2023)"},{"why":"Introduces MIWAE, the missing-data VAE baseline that APCs outperform in reconstruction experiments.","marker":"Mattei & Frellsen (2019)"},{"why":"EinsumNetworks are used as the circuit encoder structure for tabular data.","marker":"Peharz et al. (2020a)"},{"why":"Defines the VAE baseline and the approximate-posterior framework that APCs contrast with.","marker":"Kingma & Welling (2014)"}],"fun_headline_variants":["Autoencoding probabilistic circuits handle missing data","Exact inference yields robust probabilistic autoencoders","One circuit, explicit embeddings, no imputation needed","Probabilistic circuits learn embeddings via tractable inference","PC autoencoders beat neural rivals on missing data"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole result rests on the heuristic that randomly pairing each embedding input with one data input at the circuit's bottom layer produces a conditional distribution $p_{\\mathcal{C}}(Z\\mid X=x)$ informative enough for reconstruction; the paper gives no rule for how strong or deep that coupling should be.","fun_headline_variants_meta":{"raw":{"variants":["Autoencoding probabilistic circuits handle missing data","Exact inference yields robust probabilistic autoencoders","One circuit, explicit embeddings, no imputation needed","Probabilistic circuits learn embeddings via tractable inference","PC autoencoders beat neural rivals on missing data"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000221,"raw_usage":{"total_tokens":1452,"prompt_tokens":947,"completion_tokens":505,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":563,"completion_tokens_details":{"reasoning_tokens":433}},"tokens_in":563,"tokens_out":505,"duration_ms":5208,"temperature":1.0,"reasoning_tokens":433,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T19:49:20.814884+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train an APC on an image dataset with deliberately adversarial pairings, such as pairing every embedding variable with a fixed pixel or with pixels that carry no class information, and re-run the reconstruction and downstream experiments; if performance collapses to near the VAE baseline, the robustness claim depends on the random-coupling heuristic rather than on the tractable-conditional mechanism itself. Alternatively, measure the conditional mutual information $I(X;Z)$ under the learned circuit; if it is close to zero at high corruption levels while reconstruction still succeeds, the information must flow through the decoder rather than the encoder, which would contradict the paper's attribution.","supporting_citations":[],"review_version":1}