{"id":"395cffe5-87d1-4e5c-be64-0e00d3776473","arxiv_id":"2507.11136","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":8,"one_line_summary":"A fully Bayesian tensor-network kernel machine with automatic rank and feature dimension selection, trained by mean-field variational inference.","lead":"This paper introduces a Bayesian version of tensor-network kernel machines, where sparsity-inducing priors automatically select the model's rank and feature dimensions. It offers uncertainty estimates at similar computational cost to the deterministic alternating least squares algorithm, with experiments on six real-world data sets.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The Student-t predictive variance in Eq. (43) is a first-order delta-method result that drops products of factor posterior variances; the claimed well-calibrated uncertainty quantification is not established until this bias is quantified.","rationale":"The reader's weakest assumption already flagged the mean-field variational approximation plus the predictive approximation in Appendix 11 as the place where well-calibrated uncertainty and automatic complexity selection could break. My concern isolates a concrete, verifiable failure mode of exactly that assumption: the predictive variance formula in Eq. (43) is a first-order expansion in the factor posterior covariances and omits all higher-order product terms. Because all omitted terms are nonnegative, the approximate predictive variance is biased downward whenever posterior variances are not extremely small, and this is precisely the regime in which uncertainty quantification is most valuable. The bias is not an issue of external consensus or of asking for a stronger guarantee than the field usually provides; it is a property that can be checked exactly and cheaply using the model's own fitted variational distributions. I therefore do not reject the paper or change the reader's conditional verdict. The method may well be useful and empirically competitive, as the experiments suggest; the released code makes the proposed moment comparison straightforward, and the authors should either run it or qualify the uncertainty claims. The flop-complexity claim is not the main risk: the posterior covariance computation and the deterministic ALS solve both scale cubically in M_d R, so equal asymptotic cost is credible. The credibility gap is whether the produced variance is close enough to the exact variational predictive variance to support 'well-calibrated uncertainty at no extra computational cost.' This is why the conditional verdict is appropriate but not a rejection: the central algorithmic construction is plausible and reproducible, while the strongest advertised benefit depends on an unquantified approximation.","tokens_in":30331,"tokens_out":12336,"duration_ms":170140,"concrete_test":"Use the released code to train on a low-data subset (e.g., N=100 of Concrete or Energy with R_max=25, M=20). For held-out points x_i, compute the exact predictive mean and variance under the fitted q: E[f]=Σ_r∏_d μ_{d,r}, E[f^2]=Σ_{r,s}∏_d(μ_{d,r}μ_{d,s}+C_{d,rs}), and Var(y)=Var(f)+E_q[1/τ] with E_q[1/τ]=b_N/(a_N-1). Compare these with the moments implied by Eq. (43), whose variance is b_N/(a_N-1) times Σ_d g^(d)(x_i)^T Σ^(d) g^(d)(x_i). Repeat over 100 test points and several random splits. If the median relative error of the Eq. (43) variance versus the exact q-moment exceeds about 10%, the uncertainty claims must be qualified or fixed; if it is consistently below 10%, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Under the mean-field posterior, the model output is f(x)=Σ_r ∏_d a_{d,r} with a_{d,r}=φ^(d)^T w_r^(d), where the columns w_r^(d) are jointly Gaussian and independent across modes. Consequently the exact predictive second moment is E_q[f^2]=Σ_{r,s}∏_d(μ_{d,r}μ_{d,s}+C_{d,rs}) with C_{d,rs}=φ^(d)^T Cov(w_r^(d),w_s^(d))φ^(d), while E_q[f]=Σ_r∏_d μ_{d,r}. The predictive distribution in Appendix 11 / Eq. (43) instead evaluates g^(d)(x)^T Σ^(d) g^(d)(x) at posterior means, which retains only first-order terms of the form Σ_r Σ_d C_{d,rr}∏_{k≠d} μ_{k,r}^2 (plus analogous cross-column covariance terms). All products of two or more factor variances, which are nonnegative, are omitted. In the regime where UQ matters most—small N or weakly identified factors—posterior variances are not negligible relative to the means, so the reported predictive variance is systematically too small. The Student-t form also treats the factor-induced variance as a known constant before marginalizing τ. The flop-count identity with deterministic ALS is plausible; the load-bearing question is whether the uncertainty estimates are valid.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes BTN-Kernel Machines, a variational Bayesian extension of tensor-network kernel machines. A CPD (or TT) weight tensor is assigned a hierarchical sparsity-inducing prior: Gamma-distributed precision parameters act on the rank columns and on the feature rows of each factor matrix, with the aim of automatically inferring tensor rank and feature dimensions. Variational mean-field inference yields closed-form Gaussian posteriors for the factor matrices and Gamma posteriors for the precision hyperparameters, leading to alternating updates that the authors claim have the same asymptotic cost as deterministic ALS. Predictive uncertainty is summarized by a Student-t distribution. Experiments on synthetic data and several UCI data sets compare the method with GP, T-KRR, and SP-BTN in terms of RMSE, negative log-likelihood, effective rank, and effective feature dimension.","tokens_in":30653,"tokens_out":7774,"duration_ms":102267,"significance":"The paper addresses a relevant gap: deterministic tensor-network kernel machines do not provide posterior uncertainty, and existing probabilistic tensor methods often rely on restrictive diagonal covariances or manual complexity tuning. The core variational update equations, Theorems 3–6, and the moment formulas in the appendices are derived in detail and are largely internally consistent; the authors also release code for reproducing the experiments. If the predictive uncertainty were shown to be calibrated, the method would be a useful scalable Bayesian kernel machine with automatic complexity control. However, the two headline claims—uncertainty quantification at no extra cost and automatic feature-dimension inference—are currently supported only under additional approximations and threshold-dependent heuristics that are not fully acknowledged in the main text.","major_comments":[{"comment":"The predictive distribution in Eq. (43) is not an exact marginalization under the variational posterior. The appendix integrates over vec(W^(1)) while treating g^(d)(x_n) as fixed, but g^(d)(x_n) contains the remaining random factor matrices W^(k), k≠1; sequential integration of this kind is invalid. The resulting expression is a first-order delta-method approximation in which the predicted variance is built from terms g^(d)(x_n)^T Σ^(d) g^(d)(x_n) evaluated at posterior means. The exact second moment under the mean-field posterior is E_q[f^2] = Σ_{r,s} ∏_d (μ_{d,r} μ_{d,s} + C_{d,rs}), with μ_{d,r} = φ^(d)^T \\tilde w_r^(d) and C_{d,rs} = φ^(d)^T Cov(w_r^(d), w_s^(d)) φ^(d); Eq. (43) omits products of two or more posterior covariances across different factors. The sign of the omitted remainder is not controlled, so the predictive variance can be either too large or too small, and the discussion in the paper does not establish calibration. The Student-t form is also itself an approximation after integrating out τ. This issue is load-bearing because the paper's central claim is uncertainty quantification at no extra cost; please restate the derivation as a first-order approximation and provide a quantitative assessment of its bias, e.g., by comparing against Monte Carlo moments from q on small-N examples.","section":"Section 3.3 / Appendix 11"},{"comment":"The claimed automatic inference of feature dimensions M_d is not literally achieved. Algorithm 1 only prunes zero columns (rank components); rows of the factor matrices are retained throughout, and the paper itself states in Section 4.1 that strongly penalized rows become very small but not exactly zero. The experiments therefore measure M_eff using an ad hoc 0.25% variance threshold (Section 4.3), and the reported effective feature dimension depends on that threshold. This is not the same as automatically inferring a discrete feature dimension. Please either implement row pruning or explicitly redefine the claim as soft feature selection with a threshold-dependent effective dimension, and state the threshold's influence on the results.","section":"Section 4.1 / Algorithm 1"},{"comment":"The computational-complexity claim 'same computational complexity as its deterministic counterpart' is a flop-count statement only. Maintaining the full covariance matrices Σ^(d) requires O(M_d^2 R^2) memory per mode, whereas deterministic ALS stores only O(M_d R) parameters per mode; the lower-bound evaluation in Eq. (42) and the truncation checks also add overhead. For large M_d and R the memory and constant-factor differences are substantial, so the abstract's phrase 'uncertainty quantification at no extra computational cost' overstates the situation. Please qualify the claim as asymptotic flop complexity and report memory complexity separately.","section":"Section 3.4"}],"minor_comments":[{"comment":"The predictive mean is written as \\tilde y_i = ⊛_{d=1}^D \\tilde W^(d) φ_i^(d), which is an R-vector under the paper's Hadamard-product notation, not a scalar. The scalar predictive mean should be 1_R^T (⊛_{d=1}^D \\tilde W^(d) φ_i^(d)), or an explicit summation over rank components should be introduced.","section":"Eq. (43)"},{"comment":"The text calls c0=d0=g0=h0=10^{-6} 'uninformative', but the experiments use c0=10^{-5} and d0=10^{-6} for the rank hyperparameters. Please align the description and the experimental settings, or explain why this difference is immaterial.","section":"Section 3.2.5"},{"comment":"SP-BTN results are taken from the original paper rather than measured under the same train/test splits as the other methods; this should be stated more prominently as a limitation of the comparison, not only in the experimental setup paragraph.","section":"Table 4"},{"comment":"The final expression in the proof of Theorem 6 uses ⊛_{k≠d} where the context requires ⊛_{d=1}^D; the index is inconsistent with the previous line and with Eq. (40).","section":"Appendix 8, Eq. (55)"}],"recommendation":"major_revision","confidential_remarks":"The technical core of the paper—the variational ALS updates with full Gaussian posteriors—is sound and is a useful contribution to the tensor-kernel literature. The main risk is that the uncertainty-quantification claim is currently supported by an unvalidated delta-method approximation. I would require either a corrected derivation of the predictive moments or explicit calibration experiments comparing the approximate predictive intervals with Monte Carlo estimates from the variational posterior, especially in small-N settings. The feature-selection claim also needs to be softened or supported by an actual row-pruning mechanism."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear colleague,\n\nThe headline: this is a real contribution—a fully Bayesian tensor-network kernel machine with ARD priors for rank and feature shrinkage, and the mean-field updates are derived carefully using Khatri-Rao identities. The code is public and the derivations are mostly reproducible. But the paper oversells the uncertainty quantification and the 'zero extra cost' claim.\n\nWhat's genuinely new: the hierarchical ARD prior is from Zhao et al. (2015a), but applying it to Kronecker-product kernel features and deriving closed-form variational updates that exploit the Khatri-Rao structure is a solid, nontrivial extension. The experiments are fair in structure—GP, T-KRR, SP-BTN—and results are competitive, though not uniformly superior as the abstract claims.\n\nThe main soft spot is the predictive distribution. Appendix 11 marginalizes factor matrices sequentially and ends with a Student-t whose variance is Σ_d g^(d)(x)^T Σ^(d) g^(d)(x). But exact integration would also produce products of posterior covariances across modes; those terms are all nonnegative and are dropped. That makes the predictive variance systematically too small when posterior variances are non-negligible relative to the means—exactly the small-N regime where uncertainty matters. The paper does not state this approximation or quantify it. This is a load-bearing issue for the uncertainty quantification claim, though not for the predictive mean.\n\nThe complexity claim also needs qualification. The flop count may match deterministic ALS, but storing Σ^(d) costs O(M_d^2 R^2) per mode, which is much more than the O(DMR) for factor matrices alone. 'No extra computational cost' is misleading.\n\nFeature selection is also described too strongly: rows shrink but rarely hit zero, so the method does soft shrinkage and then applies a variance threshold to define effective dimensionality. That's a reasonable engineering choice, but 'automatic inference of feature dimensions' should be stated as approximate.\n\nMinor: a few typos in the appendix (e.g., Σ^(k) where Σ^(d) is meant), and the TT extension is only sketched.\n\nNet: this deserves peer review. The core method is sound and useful for the tensor-kernel community. I'd condition acceptance on the authors either improving the predictive approximation or clearly stating its first-order nature and adding a calibration check on small-N data.\n\nBest,","headline":"Solid Bayesian TN kernel machine with ARD, but the predictive variance is a first-order approximation and the zero-cost claim is overstated.","tokens_in":31156,"tokens_out":5305,"would_cite":true,"duration_ms":63914,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that a mean-field variational approximation over tensor-network factors yields a Bayesian alternating least-squares algorithm whose computational complexity matches the deterministic version, so predictive uncertainty is…","keywords":["tensor network kernel machines","variational inference","automatic relevance determination","tensor rank selection","feature selection","uncertainty quantification","Bayesian alternating least squares","Kronecker product features"],"falsifier":"Take a small tensor regression problem with strongly correlated true factor columns, run the variational algorithm and an MCMC sampler on the same model, and compare predictive interval coverage and selected rank; if the variational intervals under-cover systematically or the selected rank diverges from the MCMC posterior, the cost-free uncertainty claim fails.","tokens_in":30127,"feed_emoji":"🧮","tokens_out":8742,"duration_ms":96908,"temperature":0.7,"pith_summary":"Tensor-network kernel machines represent the weight vector of a kernel model as a low-rank tensor decomposition, which makes learning fast but leaves two problems: the weights are point estimates with no uncertainty, and the tensor rank and feature dimensions must be tuned by hand. This paper proposes a fully Bayesian version in which every factor matrix carries a Gaussian prior and the precisions of those priors are themselves random variables with Gamma hyperpriors. The paper's central claim is that a mean-field variational approximation on the tensor-network factors produces a Bayesian alternating least-squares algorithm with the same computational complexity as its deterministic counterpart, so uncertainty quantification comes at no additional asymptotic cost. Under this model, the hierarchical priors shrink entire rank components and irrelevant feature rows toward zero, so tensor rank and feature dimensions are inferred from data. Experiments on synthetic and real datasets are presented as evidence that the method recovers ground-truth complexity, gives calibrated predictive intervals, and scales to a dataset with 45,222 samples and 96 features.","feed_headline":"Bayesian tensor-network kernels add uncertainty at zero extra cost","feed_subtitle":"Posterior updates keep the deterministic ALS complexity, so calibrated predictive intervals cost nothing extra.","key_machinery":"The machinery is a conjugate mean-field variational Bayes algorithm over a CP-decomposed weight tensor. Factor matrices $W^{(d)} \\in \\mathbb{R}^{M_d \\times R}$ are assigned zero-mean Gaussian priors with precision $\\Lambda_R \\otimes \\Lambda_{M_d}$; the shared precision vector $\\lambda_R$ prunes whole rank columns, $\\lambda_{M_d}$ prunes feature rows, and Gamma hyperpriors on both implement automatic relevance determination. The posterior covariance $\\Sigma^{(d)}$ in Eq. (21) is the load-bearing object: its diagonal generates the variance terms feeding the hyperparameter and predictive updates, and the reshaping identities in Theorems 4 and 6 express second-moment terms as Khatri-Rao and Hadamard products of already-computed matrices. This preserves the exact ALS structure in the mean update while adding covariance terms, and integrating out the noise precision yields a Student's $t$ predictive distribution.","core_discovery":"The central discovery is a conjugation structure: when the weight vector of a kernel machine is CP-decomposed as $w = \\sum_r w_r^{(1)} \\otimes \\cdots \\otimes w_r^{(D)}$, placing zero-mean Gaussian priors on the factor matrices with Kronecker precisions $\\Lambda_R \\otimes \\Lambda_{M_d}$, and Gamma hyperpriors on those precisions, makes the variational posterior updates close under the mean-field factorization. The update for each factor matrix takes the form of an alternating least-squares step, with the posterior mean given by the same Khatri-Rao and Hadamard design matrices used by deterministic ALS, and the posterior covariance $\\Sigma^{(d)}$ carrying the additional uncertainty terms. Those covariance terms propagate into Gamma updates for the shared column-precision $\\lambda_R$ and the per-mode row-precision $\\lambda_{M_d}$, giving automatic rank and feature selection, and into a Student's $t$ predictive distribution for new targets. The paper's headline result is that all of this happens at the same $O\\left(\\sum_d N M_d^2 R^2 + M_d^3 R^3\\right)$ per-iteration complexity as deterministic ALS, so the Bayesian machinery is not paid for in asymptotic runtime.","pith_inferences":["Because the posterior covariance is already computed for every factor matrix, an active-learning or Bayesian-optimization extension could query points using the predictive variance at essentially no additional per-iteration cost; the paper does not pursue this use.","The negative correlation between the $\\lambda_R$ and $\\lambda_{M_d}$ updates means a heavily penalized rank component can be offset by relaxing row precisions, a compensation mechanism that could be isolated by ablating either hyperprior.","The same ARD-style hierarchical prior should transfer to tensor-train cores, and a full TT implementation would be a direct test of whether automatic multi-rank inference behaves as well as the CPD case; the paper only sketches this extension.","If the no-extra-cost claim survives at scale, deterministic tensor-kernel pipelines in production could be upgraded to output calibrated intervals and feature relevance without changing their training-time complexity; this is an inference from the paper's result, not a claim it makes."],"forward_implications":["Uncertainty quantification becomes a free add-on to tensor-network kernel machines, with the same asymptotic per-iteration cost as deterministic ALS.","Tensor rank and feature dimensions no longer need manual cross-validation; the hierarchical priors shrink unnecessary components to zero during training.","The per-mode precision vectors act as feature-relevance scores, so the trained model doubles as an interpretability tool.","The method scales to large high-dimensional datasets: in the paper's experiments, it handles a 45,222-sample, 96-feature classification task and matches or improves on Gaussian-process RMSE and NLL on several benchmarks.","A tensor-train variant follows from the same construction by giving each TT rank its own precision vector, as Section 3.5 sketches."],"supporting_citations":[{"why":"Provides the variational inference framework and the optimal $q_j$ update rule that the paper uses to derive all posterior updates.","marker":"Bishop (2006)"},{"why":"Supplies the Bayesian CP model with ARD-style automatic rank determination whose hierarchical prior the paper adapts to tensor-network kernel machines.","marker":"Zhao et al. (2015a)"},{"why":"Introduces the automatic relevance determination framework that motivates the sparsity-inducing hierarchical priors.","marker":"Neal (1996)"},{"why":"Defines the tensor-train decomposition and its rank structure, which the paper contrasts with CPD and extends in Section 3.5.","marker":"Oseledets (2011)"},{"why":"Sets out the Gaussian process baseline and the $O(N^2)$ memory, $O(N^3)$ flop scaling that motivates low-rank kernel approximations.","marker":"Rasmussen and Williams (2006)"},{"why":"Provides the deterministic tensor-kernel ridge regression baseline and the polynomial and Fourier feature construction used in the experiments.","marker":"Wesel and Batselier (2021)"},{"why":"Supplies the structured-posterior Bayesian tensor network baseline whose uncertainty limitations the paper compares against.","marker":"Konstantinidis et al. (2022)"}],"fun_headline_variants":["Bayesian tensor kernels: uncertainty without extra compute","Auto-tuned tensor kernels learn rank and features","Tensor kernel machines now quantify uncertainty for free","Bayesian tensor nets: same speed, built-in uncertainty","Interpretable Bayesian kernel machines auto-select features"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the mean-field approximation, which treats all factor matrices as independent in the posterior, is accurate enough that the Student's $t$ predictive intervals and the automatically selected tensor ranks remain well calibrated.","fun_headline_variants_meta":{"raw":{"variants":["Bayesian tensor kernels: uncertainty without extra compute","Auto-tuned tensor kernels learn rank and features","Tensor kernel machines now quantify uncertainty for free","Bayesian tensor nets: same speed, built-in uncertainty","Interpretable Bayesian kernel machines auto-select features"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00022,"raw_usage":{"total_tokens":1472,"prompt_tokens":994,"completion_tokens":478,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":610,"completion_tokens_details":{"reasoning_tokens":421}},"tokens_in":610,"tokens_out":478,"duration_ms":5803,"temperature":1.0,"reasoning_tokens":421,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T17:16:05.337229+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a small tensor regression problem with strongly correlated true factor columns, run the variational algorithm and an MCMC sampler on the same model, and compare predictive interval coverage and selected rank; if the variational intervals under-cover systematically or the selected rank diverges from the MCMC posterior, the cost-free uncertainty claim fails.","supporting_citations":[],"review_version":1}