{"id":"14cc76b8-21e4-4bcb-8922-182eabc49290","arxiv_id":"2608.12869","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Each sum node's contribution to the Hessian trace of a probabilistic circuit equals its flow squared times a local curvature term, and gating regularization by the local term fixes the underfitting caused by global trace regularization.","lead":"Probabilistic circuits are generative models that allow exact measurement of loss-surface sharpness. This paper shows a node's sharpness contribution factorizes into usage and local curvature, and uses that split to build a regularizer that avoids the underfitting caused by uniform sharpness penalties.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Algorithm 2 computes the gate from squared dataset-averaged ratios, not from the empirical local trace defined in Eq. (6); the reported gated results may not test the paper's proposed local-curvature mechanism.","rationale":"I agree with the reader's CONDITIONAL verdict. The central theoretical result, Theorem 1, is an exact algebraic factorization that follows directly from the edge-flow identity F_nc/theta_nc = (p_c/p_n)F_n; I see no error in the decomposition or in Propositions 1, 2, and Corollary 1. The depth-bias analysis is also sound, though its quantitative claims rely on the typical assumption rho < 1, which can fail in deterministic branches. The primary problem is the gate mismatch: Algorithm 2 defines a different empirical local trace than Eq. (6), meaning the implemented regularizer may not be the one motivated by the theory. This is not cosmetic; it changes which nodes receive stronger penalization and can reverse rankings, so the empirical support for 'adaptive sharpness-aware learning' is currently ambiguous. A secondary issue is that the supplement's proof of Proposition 6 contains an algebraic inconsistency: the derivative of the penalty term omega_n * sum_c (N_nc/theta_nc)^2 is written as mu * omega_n * N_nc/theta_nc^2, which is neither the correct derivative (it should have N_nc^2/theta_nc^3) nor consistent with the printed quadratic. This may stem from an unstated surrogate inherited from prior work, but it needs clarification. Both concerns are fixable and do not touch the correctness of Theorem 1, so the paper remains promising. However, until Algorithm 2 is corrected (or the implementation confirmed to match Eq. (6)), the reported gated gains cannot be attributed to local-curvature gating as defined. Thus I recommend keeping the CONDITIONAL verdict, with the condition that the authors fix the algorithm definition, clarify the surrogate derivation, release code, and rerun or verify the key experiments.","tokens_in":22137,"tokens_out":8755,"duration_ms":86881,"concrete_test":"Run the gated EM experiments with the gate recomputed as b_t_n = (1/N) * sum_i sum_c (p_c(x_i)/p_n(x_i))^2, the definition in Eq. (6), and compare the resulting test NLL to Table 2. Additionally, on a trained circuit, compute the Spearman rank correlation between the two gate variants. If the corrected gate yields different node rankings or changes gated test NLL materially on any dataset, the published Algorithm 2 does not implement the theory and the reported gains need re-evaluation.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The load-bearing weakness is a mismatch between the local trace defined in the theory and the quantity computed by Algorithm 2. The paper defines the empirical local trace as b_t_n = (1/N) * sum_i sum_{c in ch(n)} (p_c(x_i)/p_n(x_i))^2 (see the 'Local-Curvature Gating' paragraph and Eq. (6)). Algorithm 2, lines 9-10, instead computes \\hat t_n = sum_{c in ch(n)} ( (1/|D|) * sum_{x in D} p_c(x)/p_n(x) )^2. These are not equal: the first averages the squared ratios over samples, the second squares the sample-averaged ratio first. By Jensen's inequality, b_t_n >= \\hat t_n, and more importantly the ranking of nodes can change, so the gate omega_n = \\hat t_n / max \\hat t_n may penalize different nodes than the theory's local curvature. Since the adaptive method's stated purpose is to allocate regularization by intrinsic local curvature, the experimental results in Table 2 and Figures 7-8 cannot be interpreted as validating that mechanism unless the implementation actually uses Eq. (6)'s b_t_n. No code is released, so the reader cannot determine which definition was run; the paper must correct Algorithm 2 and verify the equivalence.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper develops a compositional theory of curvature in probabilistic circuits. It proves that, for smooth and decomposable PCs, the per-node contribution to the Hessian trace of the negative log-likelihood factorizes exactly as T_n(x) = F_n(x)^2 t_n(x), where F_n is the circuit flow through node n and t_n(x) = sum_c (p_c(x)/p_n(x))^2 is a local trace. It further shows that the local Hessian of a sum node is rank one with unique nonzero eigenvalue t_n, analyzes how flow attenuation through sum edges creates a depth bias in the global trace, and derives an exact condition for ranking reversals between global contribution and local curvature. Motivated by experiments showing that global trace regularization underfits at higher data fractions, the paper proposes a gated regularizer that scales the trace penalty by a monotone function of the empirical local trace, preserving closed-form EM updates. Experiments on 20 DEBD datasets compare unregularized, global-trace, and gated training, reporting that gated training recovers much of the fit lost by global regularization.","tokens_in":22289,"tokens_out":11746,"duration_ms":113136,"significance":"The theoretical core is sound and valuable: the factorization is exact, the rank-one local-Hessian characterization is clean, and the depth-bias analysis provides a concrete mechanism for the observed underfitting of global trace regularization. The empirical observations, including concentration of the global trace in few nodes and the failure of top-global-contribution selection, are interesting and largely consistent with the theory. However, the paper's algorithmic implementation and its derivation contain two load-bearing problems: the gate computed in Algorithm 2 does not match the local trace defined in Eq. (6), and the proof of the closed-form update contains a mathematical error. Because the experimental claims are presented as validating the local-curvature mechanism, these issues must be resolved before the paper can be accepted.","major_comments":[{"comment":"The gate implemented in Algorithm 2 computes \\hat t_n = \\sum_{c in ch(n)} ( (1/|D|) \\sum_{x in D} p_c(x)/p_n(x) )^2, whereas the paper defines the empirical local trace as \\hat b_{t_n} = (1/N) \\sum_{i=1}^N \\sum_{c in ch(n)} (p_c(x_i)/p_n(x_i))^2. These two quantities are not equal: by Jensen's inequality the latter is at least the former for each child term, and the induced node rankings can differ. Since the adaptive method is motivated as allocating regularization according to the local curvature of Eq. (6), the results in Table 2 and Figures 7-8 cannot be interpreted as validating that mechanism unless Algorithm 2 is corrected or the authors show that the two quantities produce the same node ranking on the evaluated circuits. No code is released, so the reader cannot determine which quantity was actually used. This is a load-bearing issue for the central empirical claim.","section":"Algorithm 2, lines 9-10; 'Local-Curvature Gating' and Eq. (6)"},{"comment":"The derivation of the closed-form update in Proposition 3 is not mathematically consistent. In the appendix, the Lagrangian for the regularized objective is differentiated to obtain \\partial L/\\partial \\theta_{nc} = N_{nc}/\\theta_{nc} - \\lambda + \\mu \\omega_n N_{nc}/\\theta_{nc}^2, but the correct derivative of -\\mu \\omega_n \\sum_c (N_{nc}/\\theta_{nc})^2 is N_{nc}/\\theta_{nc} - \\lambda + 2\\mu \\omega_n N_{nc}^2/\\theta_{nc}^3. The stated quadratic equation \\lambda \\theta_{nc}^2 - N_{nc}\\theta_{nc} - \\mu \\omega_n N_{nc} = 0 therefore does not follow from that Lagrangian. In addition, the main text's objective uses S_{nc} = \\sum_i F_{nc}(x_i)^2, while the appendix uses N_{nc}, and the 'surrogate used by global trace-regularized EM' is never explicitly defined. Since Eq. (7) is the proposed update rule, the authors need to provide a correct derivation or explicitly state and prove the surrogate step.","section":"Gated EM Update and Proposition 3; Appendix 'Adaptive Gated Regularization', Proposition 6"}],"minor_comments":[{"comment":"The symbol \\hat t_n is used for two different quantities: the empirical local trace in the main text and the dataset-averaged marginal-ratio summary in Algorithm 2. Distinct notation should be used to prevent exactly the confusion described above.","section":"Algorithm 2 and Eq. (6)"},{"comment":"The initialization \\omega_n \\leftarrow g(0) is undefined for the proposed gate g(\\hat t) = \\hat t / \\max \\hat t, since 0/0 is not defined. The authors should specify the value of g(0) or initialize gates after the first forward-backward pass.","section":"Algorithm 2, line 2"},{"comment":"The figure reports 'Normalised contribution (%)' for both the global and local traces, but these quantities are normalized separately on different scales; the caption should say so explicitly to avoid implying a common scale.","section":"Figure 6 caption"},{"comment":"The paper relies heavily on the authors' prior work (Suresh et al. 2026) for the global trace regularizer and the EM surrogate; the introduction should state this dependency more explicitly when framing the contribution.","section":"Introduction and references"},{"comment":"The text claims global regularization 'degrades performance on the remaining 18' datasets at the full data fraction, which is consistent with the table, but on bbc and dna the global regularizer improves substantially; a one-sentence acknowledgement of these exceptions would make the summary more precise.","section":"Q3, Table 2"}],"recommendation":"major_revision","confidential_remarks":"The theoretical decomposition and rank-one local geometry are correct and publishable, but the empirical validation is not yet trustworthy because Algorithm 2 appears to implement a different gate from the one defined in the theory, and the proof of Proposition 3 contains a derivative error. I recommend major revision, and I would be willing to re-review after the authors correct Algorithm 2, provide a valid derivation or explicit surrogate for Eq. (7), and ideally release code or state precisely which quantity was used in the reported experiments."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: the paper's theory is clean and largely correct, but its headline empirical claim is undermined by a mismatch between the gate defined in the text and the gate implemented in Algorithm 2. Worth refereeing, but the authors need to correct that before publication.\n\nWhat's actually new: the per-node factorization T_n = F_n^2 t_n, the rank-one local Hessian result, the depth-bias analysis via flow attenuation, and the ranking-reversal condition. Theorem 1 is a short consequence of the known edge-flow identity, so it's not a deep result, but it's a useful conceptual reframing: curvature is local in origin and contextual in global effect. The observation that global trace regularization underfits at high data (only 2/20 datasets improve) is a real and useful finding. The gated regularizer is a natural follow-on and preserves the closed-form EM update.\n\nThe soft spot: Algorithm 2 computes \\hat t_n = \\sum_c ( (1/N)\\sum_i p_c(x_i)/p_n(x_i) )^2, whereas the text defines the empirical local trace as b_t_n = (1/N)\\sum_i \\sum_c (p_c(x_i)/p_n(x_i))^2. These are different by Jensen; the ranking of nodes can change. Since the whole empirical section is motivated by allocating regularization according to local curvature b_t_n, the results in Tables 2 and Figures 7-8 cannot be read as validating that mechanism unless the implementation actually uses Eq (6)'s quantity. No code is released, so nobody can tell which was run. This is a fixable problem: correct Algorithm 2, re-run the experiments, or explicitly reframe the method as using \\hat t_n and give a separate justification. As is, the empirical contribution is on shaky ground.\n\nOther, smaller issues: the central trace formula (3) and the EM surrogate leading to Eq (7) are inherited from the authors' prior work without proof in this paper; that's acceptable if the citations are solid, but it does mean the novelty is mostly in the interpretation, not the machinery. Some tables lack error bars or significance tests. The writing is mostly clear, though a few passages read like footnotes in the main text.\n\nBottom line: the theory is worth knowing; the empirical method as specified is not yet verified. This deserves a serious referee, not a desk reject. I'd send it to review with a request for an explicit correction of Algorithm 2 and a re-analysis of the main experiments. If the authors can show that the rankings of \\hat t_n and b_t_n are close enough on these datasets, the paper goes through; if not, they need to re-run the experiments.","headline":"Clean theory, useful decomposition, but the headline experiments are undercut by an implementation mismatch in Algorithm 2; still worth refereeing.","tokens_in":22959,"tokens_out":5589,"would_cite":true,"duration_ms":50689,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Each sum node's curvature contribution in a probabilistic circuit splits exactly into squared flow times local sharpness, which explains why global sharpness regularization underfits and motivates a gated fix.","keywords":["probabilistic circuits","Hessian trace","sharpness-aware learning","circuit flow","local curvature","regularization","generative models","loss landscape"],"falsifier":"Take any trained smooth, decomposable PC with strictly positive outgoing sum weights and any input $x$; compute every node's $T_n(x)$, $F_n(x)^2$, and $t_n(x)$ from a single forward–backward pass and check whether $T_n(x) = F_n(x)^2 t_n(x)$ holds to numerical precision, a single violation falsifies the identity. To test the gating claim, compare the Algorithm 2 gate $\\sum_c \\bigl(\\frac{1}{N}\\sum_i p_c(x_i)/p_n(x_i)\\bigr)^2$ with the true empirical local trace $\\frac{1}{N}\\sum_i \\sum_c (p_c(x_i)/p_n(x_i))^2$ on trained circuits: if the two rank nodes differently, the reported gains are not attributable to local-curvature gating.","tokens_in":21812,"feed_emoji":"🧮","tokens_out":16721,"duration_ms":131756,"temperature":0.7,"pith_summary":"Probabilistic circuits are generative models whose structure makes the trace of the Hessian of the negative log-likelihood exactly computable, and prior work used that trace as a global sharpness regularizer to seek flatter optima. This paper proves that such global regularization is misspecified: each sum node's contribution to the trace factorizes exactly as $T_n(x) = F_n(x)^2 t_n(x)$, separating how heavily the node is used (squared circuit flow) from how sharply curved its own output distribution is (local trace). Because flow is amplified near the root and attenuated along upstream sum edges, the global trace concentrates on shallow, heavily used nodes rather than intrinsically sharp ones, which explains why a uniform penalty lowers both training and test likelihood. The paper then introduces a gated regularizer that scales the existing trace penalty by empirical local curvature while preserving closed-form EM updates, and reports that it matches or improves the unregularized model on most of the 20 DEBD benchmark datasets, where the global regularizer degrades 18.","feed_headline":"Probabilistic circuits: curvature equals usage times local sharpness","feed_subtitle":"The exact split explains why uniform sharpness penalties underfit, and a local-curvature gate restores test fit.","key_machinery":"The load-bearing object is the exact global–local factorization of the Hessian trace, $T_n(x) = F_n(x)^2 t_n(x)$, obtained by substituting the edge-flow identity $F_{nc}(x)/\\theta_{nc} = \\rho_{nc}(x) F_n(x)$, with $\\rho_{nc} = p_c/p_n$ the output ratio of child $c$ to its sum parent $n$, into the definition of the node's trace contribution. Two companion results carry the argument. Locally, the Hessian of a sum node's negative log-output with respect to its edge weights is the rank-one outer product $\\rho_n \\rho_n^\\top$, whose unique nonzero eigenvalue equals $t_n = \\|\\rho_n\\|_2^2$, so $t_n$ simultaneously is the local trace, the maximum curvature, and the total Hessian magnitude. Globally, the circuit-flow recursion is unrolled to show that in a tree, flow through a node is the product of routing responsibilities over upstream sum edges only, which yields the geometric attenuation bound $T_n(x) \\le \\rho^{2 d_\\Sigma(n)} t_n(x)$ and the depth bias of the global trace. The algorithmic payoff is the gated EM update $\\theta_{nc} = (N_{nc} + \\sqrt{N_{nc}^2 + 4\\lambda_n \\mu \\omega_n N_{nc}})/(2\\lambda_n)$, identical in form to the global trace update with the node-specific strength $\\mu\\omega_n$.","core_discovery":"On the paper's own terms, the central discovery is an exact identity: in any smooth, decomposable probabilistic circuit with strictly positive sum weights, the contribution $T_n(x)$ of a sum node $n$ to the Hessian trace of the negative log-likelihood factorizes as $T_n(x) = F_n(x)^2 t_n(x)$, where $F_n$ is the circuit flow through $n$, the sensitivity of the root probability to the node's output, and $t_n(x) = \\sum_c (p_c(x)/p_n(x))^2$ is a purely local sharpness term determined by the node's own mixture. Summing over sum nodes recovers the total trace, $\\mathrm{Tr}(\\nabla^2_\\theta \\ell) = \\sum_n F_n(x)^2 t_n(x)$. Around this identity the paper builds a full characterization: the local Hessian of a sum node is rank one with unique nonzero eigenvalue exactly $t_n$; in a tree-structured circuit flow is the product of routing responsibilities along upstream sum edges, so the global trace is depth-biased and locally sharp deep nodes are discounted geometrically in the number of upstream sum edges; and two nodes' global rankings reverse precisely when $t_i/t_j > (F_j/F_i)^2$, so the most globally curved nodes need not be the locally sharpest ones. These results motivate the paper's proposal: keep the trace penalty but gate it per node by the empirical local curvature through $\\omega_n = \\hat{t}_n / \\max_m \\hat{t}_m$, which preserves the closed-form EM update with node-specific strength $\\mu\\omega_n$ and, empirically, recovers the fit that uniform global regularization sacrifices.","pith_inferences":["The same decomposition suggests curvature-aware compression: nodes low in both flow and local curvature contribute negligibly to the trace, so measuring $F_n^2$ and $t_n$ separately could reveal which mixture components can be pruned without touching the structurally load-bearing ones.","Because the identity holds per input, the measurement pipeline could also support targeted robustness analysis, for example flagging locally sharp but low-flow nodes that have little influence on typical inputs yet might destabilize under distribution shift.","The paper itself notes that its simple gate is a proof of concept and that any monotone gate preserves the closed-form update; a direct testable extension is a gate that jointly accounts for local geometry and contextual usage, such as $F_n^2 t_n$ or a rank-based variant.","A natural check of the adaptive method's mechanism is whether gating on the true per-sample average of squared output ratios, rather than the square of the dataset-averaged ratio used in the paper's Algorithm 2, changes which nodes get regularized and by how much."],"forward_implications":["Global sharpness regularization is depth-biased: because flow is attenuated along upstream sum edges, the trace penalty lands preferentially on shallow, heavily used nodes, and uniform application can lower both training and test likelihood even as the model becomes flatter.","The global trace contribution $T_n$ and the local curvature $t_n$ induce different node rankings, with a reversal exactly when $t_i/t_j > (F_j/F_i)^2$; selecting nodes by their global contribution can therefore worsen performance, as the experiments observe.","Local curvature $t_n$ alone determines a sum node's entire ambient second-order geometry, since its local Hessian is rank one with eigenvalue $t_n$, so it is a sound basis for allocating regularization.","A locality-gated trace regularizer preserves the closed-form, linear-time EM update, recovering the unregularized model's fit on most benchmarks while keeping the low-data generalization gains of sharpness-aware training.","In the high-data regime the gated method matches or beats the unregularized baseline on the majority of the 20 DEBD datasets, whereas global trace regularization improves only two and degrades the remaining eighteen."],"supporting_citations":[{"why":"Supplies the exact Hessian-trace computation for PCs (trace equals sum of squared edge gradients) and the global trace-regularized EM update that this paper decomposes and modifies.","marker":"(Suresh et al. 2026)"},{"why":"Supplies the circuit-flow recursion, the forward–backward pass, and the EM view of edge flows that Theorem 1's factorization and the gated update rely on.","marker":"(Liu and den Broeck 2021)"},{"why":"Provides the smooth/decomposable probabilistic-circuit formalism and the tractability guarantees that define the setting of the theorem.","marker":"(Choi, Vergari, and den Broeck 2020)"},{"why":"Introduces sharpness-aware minimization, the generalization rationale that the paper refines from a global into a local, node-wise regularizer.","marker":"(Foret et al. 2021)"},{"why":"Provides the scalable circuit implementation and training harness used for the empirical comparisons.","marker":"(Liu, Ahmed, and den Broeck 2024)"},{"why":"Defines the 20 DEBD benchmark suite on which the generalization and underfitting results are measured.","marker":"(Van Haaren and Davis 2012; Bekker et al. 2015)"}],"fun_headline_variants":["Probabilistic circuits: curvature factorizes into flow and local sharpness","Why global sharpness penalties underfit: depth-biased curvature","Adaptive sharpness gate restores generalization in PCs","Curvature equals flow squared times local sharpness"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The reported gains of the adaptive regularizer rest on the assumption that the gate it computes in Algorithm 2, the square of the dataset-averaged child-to-parent output ratio, faithfully tracks the true average local curvature, even though the two quantities are not mathematically the same.","fun_headline_variants_meta":{"raw":{"variants":["Probabilistic circuits: curvature factorizes into flow and local sharpness","Why global sharpness penalties underfit: depth-biased curvature","Adaptive sharpness gate restores generalization in PCs","Curvature equals flow squared times local sharpness"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000343,"raw_usage":{"total_tokens":1951,"prompt_tokens":1079,"completion_tokens":872,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":695,"completion_tokens_details":{"reasoning_tokens":803}},"tokens_in":695,"tokens_out":872,"duration_ms":8107,"temperature":1.0,"reasoning_tokens":803,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T21:37:14.306944+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take any trained smooth, decomposable PC with strictly positive outgoing sum weights and any input $x$; compute every node's $T_n(x)$, $F_n(x)^2$, and $t_n(x)$ from a single forward–backward pass and check whether $T_n(x) = F_n(x)^2 t_n(x)$ holds to numerical precision, a single violation falsifies the identity. To test the gating claim, compare the Algorithm 2 gate $\\sum_c \\bigl(\\frac{1}{N}\\sum_i p_c(x_i)/p_n(x_i)\\bigr)^2$ with the true empirical local trace $\\frac{1}{N}\\sum_i \\sum_c (p_c(x_i)/p_n(x_i))^2$ on trained circuits: if the two rank nodes differently, the reported gains are not attributable to local-curvature gating.","supporting_citations":[{"cited_title":"Tractable Regularization of Probabilistic Circuits , year =","cited_arxiv_id":null,"evidence_quote":"Supplies the circuit-flow recursion, the forward–backward pass, and the EM view of edge flows that Theorem 1's factorization and the gated update rely on."},{"cited_title":"Sharpness-aware Minimization for Efficiently Improving Generalization , year =","cited_arxiv_id":null,"evidence_quote":"Introduces sharpness-aware minimization, the generalization rationale that the paper refines from a global into a local, node-wise regularizer."},{"cited_title":"Image Inpainting via Tractable Steering of Diffusion Models , year =","cited_arxiv_id":null,"evidence_quote":"Provides the scalable circuit implementation and training harness used for the empirical comparisons."},{"cited_title":"AAAI Conference on Artificial Intelligence, 2012 , title =","cited_arxiv_id":null,"evidence_quote":"Defines the 20 DEBD benchmark suite on which the generalization and underfitting results are measured."}],"review_version":1}