{"id":"f9e196ae-7ac3-4802-9071-859f82090ff2","arxiv_id":"2608.09117","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"A probabilistic circuit's internal node likelihoods define a pseudo-metric (HLD) whose mean and covariance can be computed exactly from the circuit, enabling a batch-level out-of-distribution test with an analytic threshold.","lead":"The paper builds a distance between probability distributions using the internal node likelihoods of a probabilistic circuit, and uses it to check whether a batch of new data comes from the same distribution as the training data. It shows the needed statistics can be computed exactly from the trained model, so no held-out calibration data is needed at deployment.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The claimed exact covariance computation may require a product operation over circuits with different scopes, which is not defined for the paper's PC class; if this fails, the no-held-out-data threshold collapses.","rationale":"The reader's weakest assumption was model misspecification (PC fidelity to the data), an acknowledged limitation that affects practical utility but not the internal correctness of the proposed computation. I identify a more fundamental, unacknowledged gap: the proof and algorithm for exact covariance computation assume the PC product operation can combine circuits with different scopes. Under the standard definition of structured-decomposable PCs and the paper's restricted input nodes, this operation is not generally available without representing the constant-1 function, which is not in the model class. If true, the central claim 'no held-out in-distribution data required' fails because Σ_P cannot be computed exactly from the trained circuit, and the proposed decision threshold lacks a principled basis. This is a correctness risk in the core methodology, not just an applicability caveat. However, I am not certain the gap is real, because some circuit implementations do support multiplication of compatible circuits over nested scopes via specialized operations; the paper does not provide enough detail or code to confirm. The concrete test above settles the issue: if Algorithm 1 reproduces the brute-force moment on a small case, the concern is resolved and the paper's conditional verdict can stand; if not, the central claim should be rejected or substantially revised. I therefore maintain a CONDITIONAL recommendation, but with a different and more load-bearing condition than the reader's.","tokens_in":56104,"tokens_out":20928,"duration_ms":209446,"concrete_test":"Build a structured-decomposable PC over three binary variables with v-tree ((X1,X2),X3): root = product(sum(A) over {X1,X2}, leaf B over X3). Select nodes with scopes {X1} and {X3} (disjoint but not siblings). Compute the covariance entry E[ℓ_{X1} ℓ_{X3}] twice: (1) exactly by enumerating all 8 joint assignments of the root distribution; (2) via Algorithm 1 as written, constructing p_{root} ⊗ p_{X1} ⊗ p_{X3} and summing over the root scope. If the results differ, the exact-computation claim is invalid. Repeat for a nested pair, e.g., {X1} and {X1,X2}. A correct implementation should match the enumeration exactly.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim that the goodness-of-fit threshold can be derived from the trained PC alone rests on the exact computation of Σ_P described under 'Exact computation of Σ_P'. The paper says: 'By Lemma 2, the scopes Si and Sj are either identical, nested, or disjoint... the corresponding subcircuits are compatible and the product operation can be applied successively to construct pSij ⊗ pi ⊗ pj.' This assumes the tractable PC product operation (Vergari et al. 2021) can multiply circuits with different scopes. Under the paper's own definition (Input nodes are univariate distributions; sum weights sum to 1; every node is a normalized distribution over its scope), a circuit over a strict subset Si cannot be lifted to a superscope Sij by multiplying with an identity, because the constant-1 function over Sij\\Si is not representable as a smooth decomposable PC with normalized leaves. In the disjoint case where Si and Sj are not siblings in the v-tree, Sij contains variables outside Si ∪ Sj, and p_{Sij} ⊗ (p_i ⊗ p_j) again requires lifting a sub-scope circuit. Algorithm 1 line 11 is therefore not well-defined as written for such scopes. This is load-bearing: without exact Σ_P, the analytic threshold (Algorithm 2) cannot be computed without held-out ID samples, defeating the paper's main contribution.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces the Hierarchical Likelihood Vector (HLV), whose coordinates are likelihoods evaluated at selected nodes of a probabilistic circuit (PC), and the Hierarchical Likelihood Distance (HLD), defined as the Euclidean distance between expected HLVs under two distributions. It proves that HLD is an integral probability metric over the linear span of the node likelihood functions, and it develops a batch-level goodness-of-fit test for unsupervised OOD detection. The key claimed contribution is that, for smooth and structured-decomposable PCs over discrete variables, the population mean and covariance of the HLV can be computed exactly from the trained PC, yielding an approximate analytic decision threshold that requires no held-out in-distribution data. Experiments on five tabular datasets, binarized MNIST at 7x7 and 28x28, node-type ablations, architecture ablations, and a localization analysis are reported.","tokens_in":56308,"tokens_out":17110,"duration_ms":191164,"significance":"If the exact moment-computation claim is made rigorous, the paper makes a valuable contribution: it turns the internal hierarchy of a PC into a statistically interpretable representation, gives an IPM interpretation of the resulting distance, and removes the need for an ID calibration sample at deployment. The theoretical components that are fully specified—Theorem 1, the CLT/quadratic-form asymptotics, the same-scope product computations, and the IPM bound in the supplement—are standard and check out. The experimental study is extensive, with 500 Monte Carlo trials, per-pair MNIST tables, and explicit treatment of model misspecification. The paper is also candid about its two main limitations: the null hypothesis is calibrated with respect to the trained model rather than the true data distribution (FPR_data reaches 0.823 at T=1000 on 28x28 MNIST), and the closed-form threshold rests on a two-moment approximation to a generalized chi-square. The main technical gap is in the exact covariance construction, which is repairable but currently not well-defined as written.","major_comments":[{"comment":"The expression pSij ⊗ pi ⊗ pj is not a well-defined product operation under the manuscript's own PC definitions. The tractable product operation cited from Vergari et al. (2021) applies to compatible circuits over a common variable scope, whereas pSij, pi, and pj generally have different scopes. Lemma 2 (the scope trichotomy) only classifies the relations among Si and Sj; it does not state that circuits over different scopes are compatible operands for a product, and the paper gives no lemma defining a cross-scope product. Moreover, because every node of the PC is defined as a normalized distribution over its scope, the constant-1 function on Sij \\ Si is not representable as a normalized input distribution for a categorical variable with more than one outcome, so the standard 'pad with an identity' lifting of a sub-scope circuit is unavailable under the paper's node semantics. This gap is load-bearing: ΣP and tr(ΣP^2) feed directly into the decision threshold in Algorithm 2, so without a valid construction of the second moments the 'no held-out ID data' contribution collapses. The issue is repairable within the same toolkit: for disjoint scopes, marginalize the root PC to S_i ∪ S_j rather than to the LCA scope S_ij; the induced v-tree on S_i ∪ S_j has root children Si and Sj, so pi ⊗ pj is a compatible same-scope circuit and the standard product operation applies. For nested scopes, first compute the same-scope product pSj ⊗ pj, marginalize the result to Si, and then multiply by pi. The authors should either rewrite Algorithm 1 along these lines or provide a precise lemma defining the cross-scope product they intend.","section":"Exact computation of ΣP; Algorithm 1, line 11"}],"minor_comments":[{"comment":"The threshold expression is presented as a normal approximation to a generalized chi-square distribution, and the paper notes that the approximation can be poor when a few eigenvalues dominate. The experimental FPR_model values support the approximation in the tested settings, but the abstract and introduction should state more explicitly that the claimed calibration is with respect to the distribution represented by the trained PC, not the true data distribution; the 28x28 MNIST FPR_data result (Table 20) makes this distinction essential.","section":"Goodness-of-Fit Test and Algorithm 2"},{"comment":"The captions of Tables 12-15 appear to be swapped. Table 12, labeled FPRdata, shows values that remain near 0.03 across all T at α=0.01, which is the behavior of the PC-vs-PC model null; Table 13, labeled FPRmodel, grows from 0.027 to 0.394 as T increases, which is the behavior of FPRdata. The same pattern appears for Tables 14 and 15 at α=0.1. Please verify the labels and correct them, since these tables directly support the calibration claims.","section":"Tables 12-15"},{"comment":"The main text's exact-covariance paragraph refers to 'Lemma 2' for the scope trichotomy, but the only lemma stated in the main text is Lemma 1; the supplement labels the scope trichotomy as Lemma 2. Renumber or cross-reference consistently.","section":"Lemma numbering"},{"comment":"Algorithm 1 uses the notation pSi ⊗ pni and pSij ⊗ pni ⊗ pnj without defining the ⊗ operation in the Background section. Since the operation is central to the exact-computation claim, a formal definition or a pointer to the precise definition in the cited composition atlas should be added.","section":"Notation for product operation"},{"comment":"The paper does not mention code or data-release plans. Given the number of reported tables and ablations, a public implementation would substantially aid reproducibility and would let readers verify the exact moment computations on real circuits.","section":"Reproducibility"}],"recommendation":"major_revision","confidential_remarks":"The product-scope gap in the exact covariance computation is the only technical blocker I found, and it is fixable: the authors can replace the LCA-scope construction by marginalizing to the union scope and then applying the standard same-scope product operation, with a separate nested-scope case. If the revised Algorithm 1 and its proof are provided, I would support acceptance. The table-caption swaps and the calibration-versus-data-distribution phrasing should also be corrected before publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague — here's my read on arXiv:2608.09117. The core idea is solid and new: represent each sample by the likelihoods of selected PC nodes (HLV), compare distributions by the Euclidean distance of expected HLVs, and use the PC itself to compute the null threshold without held-out ID data. The IPM interpretation in Theorem 1 is correct (a definitional restatement, but fine). The experiments are extensive on tabular and small MNIST, and the paper is honest about the model-misspecification problem, which shows up clearly at 28x28.\n\nThe soft spot is in the exact covariance computation, Section \"Exact computation of Σ_P\" and Algorithm 1. For covariances, you need E[ℓ_i ℓ_j] = Σ p_{S_ij} p_i p_j. When S_i and S_j are different scopes, you multiply a circuit over S_ij with circuits over subsets. That requires lifting p_i to S_ij by inserting a constant-1 factor over the extra variables. Under the paper's own definitions — input nodes are univariate distributions, every node normalized — that constant-1 factor isn't representable in the PC class. So the product operation is not well-defined for these operand scopes, and the analytic threshold, which is the main contribution, rests on an invalid step. The mean computation (µ_P) is fine because both operands have the same scope. The nested and disjoint cases both need this lift; only equal scopes avoid it.\n\nThis isn't a nitpick. If the exact Σ_P can't be computed as claimed, the no-held-out-data threshold collapses, because you'd have to fall back on sampling or a reference set. The stress-test note I got makes exactly this point, and I think it lands.\n\nThe paper does a lot right: the HLV construction is cheap, the node-type ablation is informative, and the localization figures are a nice addition. The writing is clear and the limitations section is upfront. But the central algorithmic claim needs a fix. Possible workarounds: extend the PC class to allow unnormalized leaf functions for the product operation, or restrict selected nodes to those where pairwise scopes are either equal or siblings in the v-tree (so S_ij = S_i ∪ S_j). Either way, the authors need to show the moment computation is actually tractable for their chosen node set.\n\nFor the record: I'd send this to peer review, because the question is important and the paper is close to valuable. It's not ready as-is. My overall: serious thinker yes, reading group maybe, I wouldn't cite it in its current form.","headline":"HLD is a genuinely new PC-based OOD test, but the exact covariance computation as written requires lifting sub-scope circuits with constant-1 factors that the paper's PC class cannot represent, undermining the no-held-out threshold claim.","tokens_in":56874,"tokens_out":11361,"would_cite":false,"duration_ms":115223,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that the internal node likelihoods of a trained probabilistic circuit can be turned into a statistically calibrated, batch-level out-of-distribution detector that requires no held-out in-distribution data.","keywords":["probabilistic circuits","out-of-distribution detection","integral probability metric","goodness-of-fit test","hierarchical likelihood vector","distributional shift","tractable inference","structured-decomposable"],"falsifier":"Train a structured-decomposable PC on a high-dimensional distribution it fits poorly, then feed it held-out in-distribution batches of increasing size at a fixed significance level. The paper's own 28x28 MNIST experiment is exactly this test: FPR_data rises from 0.178 at 7x7 resolution to 0.823 at T=1000, showing the detector responding to model misspecification rather than true OOD, whereas a detector calibrated to the real data distribution would keep the false-positive rate near the nominal level.","tokens_in":55864,"feed_emoji":"🎯","tokens_out":5858,"duration_ms":55344,"temperature":0.7,"pith_summary":"This paper tries to establish that the hierarchical probabilistic summaries inside a probabilistic circuit are enough to build a distribution-level out-of-distribution detector without any held-out in-distribution samples. It packs node likelihoods into a Hierarchical Likelihood Vector and measures the distance between two distributions as the Euclidean distance between their expected vectors. The paper proves this distance is an integral probability metric over a function class naturally induced by the circuit, and shows the population mean and covariance of the vector can be computed exactly and tractably for structured-decomposable circuits. That yields an approximate analytic decision threshold instead of a calibrated score, and the experiments indicate the hierarchical representation detects shifts better than root-likelihood, uncertainty, typicality, and kernel baselines while also localizing which variables changed.","feed_headline":"Internal node likelihoods alone catch out-of-distribution batches","feed_subtitle":"Comparing a circuit's expected node-likelihood vectors yields a calibrated OOD test with no held-out ID data.","key_machinery":"The central object is the Hierarchical Likelihood Vector (HLV): for a selected set of circuit nodes, each coordinate is the probability that the node's subcircuit assigns to the part of the observation lying in that node's variable scope. The argument is carried by three structural facts about smooth, structured-decomposable probabilistic circuits: the scope trichotomy lemma (any two node scopes are disjoint, equal, or strictly nested), the tractable product operation for compatible PCs, and exact marginalization. Together they let the paper compute $\\mathbb{E}_P[\\mathrm{HLV}]$ and its covariance by forming product circuits and marginalizing, with worst-case $O(S N_r^2)$ and $O(S^2 N_r^3)$ costs; a two-moment Gaussian approximation to a generalized chi-square distribution then turns the traces into a closed-form threshold.","core_discovery":"The paper's central claim is that the hierarchical probabilistic summaries inside a trained probabilistic circuit can be converted into a principled distribution-comparison test. The Hierarchical Likelihood Distance (HLD), defined as the Euclidean distance between the expected Hierarchical Likelihood Vectors of two distributions, is an integral probability metric over the function class of linear combinations of node likelihood functions. For smooth, structured-decomposable PCs, the population mean vector and covariance matrix of the HLV, and hence the traces needed for a decision threshold, are computable exactly and tractably from the circuit itself. Consequently, a trained PC alone serves as the in-distribution representation for a batch-level goodness-of-fit test whose null hypothesis is that the test batch comes from the distribution represented by the PC.","pith_inferences":["The paper's own 28x28 MNIST result, where FPR_data rises to 0.823 at T=1000, shows that the method becomes a detector of model misspecification when the circuit fits poorly; I infer that any practical deployment should validate the PC's fit before trusting the nominal significance level.","The same exact moment-computation machinery could extend to other tractable probabilistic models that support product and marginalization operations, and to continuous variables if node densities are bounded.","The node-scope ranking could be developed into an interpretable drift-localization tool for monitoring high-dimensional data, although the paper presents it only as qualitative localization cues.","Whenever the covariance spectrum is concentrated, the two-moment Gaussian approximation is the main statistical weak point; computing the generalized chi-square quantile numerically from the exactly obtained eigenvalues is a drop-in replacement."],"forward_implications":["At deployment the trained PC alone represents the in-distribution, so no held-out ID calibration or reference samples are needed for the HLD test.","Because HLD is an integral probability metric, the detector compares distributions through a probabilistically interpretable function class rather than an ad hoc feature embedding.","The exact population moments yield an approximate analytic threshold from the traces of the HLV covariance, and the approximation is accurate when no small number of eigenvalues dominates the covariance spectrum.","On the evaluated tabular datasets and 7x7 MNIST, batch-level detection reaches near-perfect power at modest batch sizes, and the node-wise decomposition of the statistic localizes distribution shifts to variable subsets.","Using leaf and sum nodes in the HLV gives the most robust detection across datasets; adding product nodes did not improve empirical detection power.","The test is calibrated with respect to the distribution represented by the trained PC, so its false-positive rate on real in-distribution data directly reflects how faithfully the PC models the underlying data distribution."],"supporting_citations":[{"why":"Supplies the tractable product operation for compatible structured-decomposable PCs that the exact moment computations rely on.","marker":"Vergari et al. 2021"},{"why":"Defines MMD, the kernel-based two-sample baseline HLD is compared against.","marker":"Gretton et al. 2012"},{"why":"Provides the multivariate central limit theorem used to derive the asymptotic null distribution of the HLD statistic.","marker":"Van der Vaart 1998"},{"why":"Gives the distribution of quadratic forms in normal variables that underlies the generalized chi-square null distribution.","marker":"Imhof 1961"},{"why":"Justifies the two-moment Gaussian approximation to the generalized chi-square quantile when no few eigenvalues dominate.","marker":"Billingsley 1995"},{"why":"Provides the Hidden Chow-Liu Tree architecture used as the probabilistic circuit in the experiments.","marker":"Liu and Van den Broeck 2021"},{"why":"Defines the typicality baseline and motivates the likelihood-paradox setting the paper addresses.","marker":"Nalisnick et al. 2019b"},{"why":"Defines Tractable Dropout Inference, the uncertainty-based baseline compared in the batch-level OOD setting.","marker":"Ventola et al. 2023"}],"fun_headline_variants":["PC node likelihoods give an OOD metric with no held-out data","HLD: an IPM from probabilistic circuit internals for OOD","No held-out data: PC's internal node likelihoods test OOD","Hierarchical likelihood distance catches OOD batches exactly","Circuit internals alone enable principled OOD detection"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The trained probabilistic circuit is an accurate enough representation of the true in-distribution that testing whether a batch comes from the circuit's distribution is a meaningful proxy for testing whether it comes from the real data distribution; if the circuit is misspecified, the test fires false alarms on genuine in-distribution data.","fun_headline_variants_meta":{"raw":{"variants":["PC node likelihoods give an OOD metric with no held-out data","HLD: an IPM from probabilistic circuit internals for OOD","No held-out data: PC's internal node likelihoods test OOD","Hierarchical likelihood distance catches OOD batches exactly","Circuit internals alone enable principled OOD detection"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000249,"raw_usage":{"total_tokens":1560,"prompt_tokens":967,"completion_tokens":593,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":583,"completion_tokens_details":{"reasoning_tokens":506}},"tokens_in":583,"tokens_out":593,"duration_ms":6090,"temperature":1.0,"reasoning_tokens":506,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T23:07:33.343891+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train a structured-decomposable PC on a high-dimensional distribution it fits poorly, then feed it held-out in-distribution batches of increasing size at a fixed significance level. The paper's own 28x28 MNIST experiment is exactly this test: FPR_data rises from 0.178 at 7x7 resolution to 0.823 at T=1000, showing the detector responding to model misspecification rather than true OOD, whereas a detector calibrated to the real data distribution would keep the false-positive rate near the nominal level.","supporting_citations":[],"review_version":1}