REVIEW 3 major objections 5 minor 38 references
Sampling from Boltzmann densities with physics informed low-rank formats
T0 review · 3 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash
Pith's one-line read A velocity field in functional tensor train format, trained on the continuity-equation residual and interleaved with Langevin and resampling steps, samples unnormalized Boltzmann densities efficiently.
desk verdict Novel TT-based Boltzmann sampler with promising numbers, but the PDE loss is not fully specified because C_t is never defined. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the functional tensor train (FTT), a tensor-train coefficient tensor contracted with univariate basis functions, here an $H^2$-orthonormalized Fourier basis. The FTT represents the velocity field $v_t$ with cores that can be optimized one at a time; for each core, the PDE residual (6) is linear in the core entries, so the update reduces to solving a small quadratic program. Two auxiliary mechanisms carry the argument: an exponential moving average of successive TT approximations with TT-SVD truncation makes the ranks adaptive, and the alternating sequence of ODE transport, resampling, and Langevin steps keeps all modes of the target populated during training and evaluation.
What would settle it
Run Algorithm 1 on a two-dimensional Gaussian mixture with two modes of equal weight whose centers are separated by a large energy barrier, starting $p_0$ concentrated on only one mode; compute the fraction of final samples in the neglected mode over many trials. The central claim predicts both modes are populated in roughly equal proportion and a small energy distance, so a reproducible run where the second mode remains empty would disprove the method's mode-coverage claim.
Extended reading notes
Core claim
The central claim is that the continuity equation governing the annealing path can be solved directly in a functional tensor train (FTT) format, and that the resulting velocity field, when evaluated by numerical ODE integration, yields accurate samples from the Boltzmann density without ever evaluating the normalization constant. Specifically, for $f_t = t f_1 + (1-t) f_0$, the authors show the continuity equation reduces to the scalar condition $f_1 - f_0 + \langle \nabla f_t, v_t\rangle - \nabla\cdot v_t + C_t = 0$, with $C_t = \partial_t \log Z_t$. The paper approximates $v_t$ as a low-rank FTT with $H^2$-orthonormalized Fourier basis functions, and trains it by minimizing the empirical squared residual (6) with ALS, where each core update is a quadratic program. Stochastic steps are added after transport: resampling reweights particle populations to handle mode-weight teleportation near $t=1$, and Langevin steps prevent collapse to point masses. The reported energy distances, for example $1.9\times 10^{-3}\pm 2\times 10^{-4}$ for the many-well $f^{4,8}_1$ with flow+stochastic, support the claim that the combination works.
Load-bearing premise
The load-bearing premise is that samples drawn from the current trained flow adequately cover the regions where the target density has mass, because the PDE residual is only evaluated on those samples; if the flow has not yet discovered a mode, the training signal never penalizes the velocity field there.
Editorial extensions
If this is right
- A user only needs the unnormalized target energy $f_1$ and an easy latent density $p_0$; no target samples and no normalizing constant are required.
- The TT ranks are not fixed in advance: the EMA update with TT-SVD truncation adapts them during training.
- The $H^2$-orthonormalized Fourier basis is what makes the PDE residual tractable; polynomial bases perform poorly, as stated in the conclusions.
- For high-dimensional many-well targets, the full pipeline flow+stochastic reaches energy distances an order of magnitude smaller than the pure TT flow, showing the stochastic steps are necessary for good mode coverage.
- Because every core update is a linear least-squares problem, training avoids the backpropagation and stability issues of physics-informed neural network solvers.
Reading between the lines
- Implicit in the paper: because the residual is evaluated on samples from the currently trained flow, the velocity field is unconstrained in regions the flow has not yet discovered; a target mode outside the initial latent support could be invisible to the PDE loss.
- A testable extension would be to add a small number of uniform or adaptive collocation points to residual (6); if this improves performance on separated modes, it would confirm that the residual's sample support, not the FTT expressivity, is the limiting factor.
- The reported experiments are two-dimensional; whether the FTT ranks remain manageable in higher dimensions is open, since TT ranks for the annealed flow can in principle grow with the complexity of the target.
- The effect of Langevin steps on the resampling proposal density $q$ is left unassessed in the paper; a correction that accounts for this bias could make the method asymptotically exact rather than approximate.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes sampling from an unnormalized Boltzmann density e^{-f_1}/Z_1 by approximating the velocity field of the continuity equation for the annealing path f_t = t f_1 + (1-t) f_0. The velocity field is represented as a functional tensor train (FTT) with an H^2-orthonormalized Fourier basis, and the coefficients are fitted by an alternating linear scheme (ALS) applied to the empirical squared residual of the logarithmic continuity equation. Training is performed on samples transported by the current approximation (ODEsolve), with optional Langevin and resampling steps inspired by sequential Monte Carlo, and TT ranks are adjusted adaptively by an exponential moving average with SVD truncation. The paper reports energy distances for Gaussian mixture and many-well targets, comparing a pure flow, a flow with post-processing, a flow with stochastic steps during training, and a stochastic-only baseline. The best reported result is an energy distance of 1.9e-3 +/- 2e-4 for the many-well problem f^{4,8}_1 with the flow+stochastic method (Table 1).
Significance. If the method works as claimed, it is a useful alternative to neural-network PDE samplers: the FTT parameterization avoids backpropagation through networks, the H^2 Fourier basis is a principled choice for PDE residuals, and the rank-adaptive ALS update is elegant. The ablation study in Table 1 separates the contributions of the flow, resampling, and Langevin steps and includes standard deviations. However, the main algorithmic claim is not yet supported because the normalizing-constant term C_t entering the objective is never specified or estimated, and because the stochastic corrections are explicitly left unanalyzed. The numerical experiments are therefore not reproducible in their current form, and the reported accuracy cannot yet be attributed to a solved continuity equation with certainty.
major comments (3)
- [Section 2, Eq. (6); Algorithm 1] The empirical loss J_{t,N} in Eq. (6) contains the term C_t = d/dt log Z_t, but the manuscript never states how C_t is computed or estimated for the unnormalized targets. Algorithm 1 lists no C_t computation or update, and the statement in Section 5 that the ground truth is known refers to validation samples, not to the training objective. This is a load-bearing omission: if C_t is replaced by a wrong constant, the residual in Eq. (3) changes by a spatially constant function, which can be absorbed by a divergent component of div v, so the learned velocity field need not solve the intended continuity equation. Until C_t is explicitly defined, estimated, and shown to enter the implemented objective, the central claim that the method solves the continuity equation in the TT format is unsupported.
- [Section 4.1] Sampling the empirical loss from the current trained flow means the PDE residual is only penalized on regions that already carry mass under the current approximation. Section 4.1 adopts this mode-seeking strategy from [19] without discussing its coverage failure mode: a target mode that is not discovered by the transported samples is never seen by the ALS residual, and no mechanism in Algorithm 1 provably discovers it. The Langevin and resampling steps are proposed as a mitigation, but their effect is heuristic. A concrete test would be to initialize from a prior that misses a well-separated mode and report whether flow+stochastic recovers it; without such a test or an analysis, the method's exploration behavior for multimodal Boltzmann densities is not established.
- [Section 4.1, Algorithm 1] The resampling step uses importance weights w^{(l)} = p(x^{(l)})/q(x^{(l)}), where q is obtained from the flow transport T_k only. Algorithm 1 nevertheless applies Langevin steps before and after resampling, and the authors explicitly state that the effect of the Langevin layers on the density q is left for future work and that only the flow layers are used. Hence the proposal density used in the weights is not the density of the samples that are actually resampled, so the stochastic correction can introduce an uncontrolled bias. Since the stochastic steps are presented as the mechanism that adjusts the relative mode weights and anneals to the correct path distribution, this limitation is load-bearing and should be addressed either by including the Langevin transition in q or by reporting a sensitivity analysis.
minor comments (5)
- [Section 2, Eq. (3)] The inner product notation in Eq. (3) is missing a comma: it should read <nabla f_t, v_t> rather than <nabla f_t v_t>.
- [Section 4.2] The UpdateTT procedure depends on an SVD truncation tolerance and an EMA weighting factor, but neither quantity is specified in Section 5, so the rank-adaptivity claim is not reproducible.
- [Section 5] The number of Langevin steps, the Langevin step size h, and the exact Fourier basis setup (n, orthonormalization domain) are not reported for each experiment, and no runtime or TT-rank trajectories are given.
- [Section 5, stochastic baseline] The description of the stochastic baseline as taking 'the ratio for the resampling step to be p_{k+1}/p_k' is ambiguous, since for Boltzmann densities this ratio contains the unknown normalizing constants Z_k/Z_{k+1}; the baseline should state explicitly whether an unnormalized density ratio is used.
- [Section 6] The claim that polynomial bases perform poorly is not supported by a dedicated experiment or a comparison in Table 1, so it reads as speculation rather than a reported finding.
Circularity Check
No significant circularity: the continuity-equation residual and the energy-distance evaluation are independent; C_t is a completeness gap, not a circular fit.
full rationale
The derivation chain is not circular. The continuity-equation residual in Eq. (3) is derived from the annealing path ft = t f1 + (1-t) f0 and the Boltzmann density definition, and Eq. (6) is an empirical least-squares version of that residual. The reported metric, energy distance between drawn samples and ground-truth samples in Table 1, is external to the training loss and is not a fitted parameter. No equation identifies a predicted quantity with an input: the velocity field is optimized with ALS, samples are transported by ODEsolve and optionally resampled/Langevin-corrected, and the metric is evaluated on those samples. The self-citations ([6], [15], [22], [33]) are used as background, for the annealing-path/teleportation motivation, and for Langevin step convergence; they are not used to define the reported predictions or to forbid alternative constructions. The reviewer's concern that C_t = d/dt log Z_t is never computed is an implementational/completeness gap in the stated objective, not a circular reduction: C_t is a term in the PDE, not a fitted version of the final energy distance, and the paper does not claim to fit C_t from the metric. The acknowledged limitations, such as leaving the effect of Langevin layers on q for future work and lacking ground-truth-free tuning, weaken guarantees but do not make the derivation self-referential.
Assumptions & free parameters
free parameters (6)
- TT rank truncation threshold (SVD tolerance) in UpdateTT =
not reported
- EMA weighting factor in UpdateTT =
not reported
- Number of Fourier basis coefficients n =
n <= 10 for problems 1 and 3; n = 41 for problem 2
- Langevin step size h and number of steps =
not reported
- Time discretization M =
M <= 35
- Basis orthonormalization domain =
[-5,5]^d for problems 1 and 3; [-50,50]^2 for problem 2
assumptions (5)
- standard math Existence of a velocity field satisfying the continuity equation along the annealing path (Eq 3)
- domain assumption Training samples drawn from the current flow cover the support of the true path distribution
- domain assumption Langevin dynamics converge asymptotically to the target for the given energies
- ad hoc to paper The H2 orthonormal Fourier basis accurately represents the velocity field for these targets
- domain assumption Resampling density q is computed from flow layers only, ignoring Langevin modifications
Cite this review
Pith. "Pith review of Sampling from Boltzmann densities with physics informed low-rank formats." pith.science (2026). https://pith.science/paper/K5CGAMBX
@misc{pith2026241207637,
author = {Pith},
title = {Pith review of: Sampling from Boltzmann densities with physics informed low-rank formats},
year = {2026},
howpublished = {\url{https://pith.science/paper/K5CGAMBX}},
note = {Machine review of arXiv:2412.07637}
}
read the original abstract
Our method proposes the efficient generation of samples from an unnormalized Boltzmann density by solving the underlying continuity equation in the low-rank tensor train (TT) format. It is based on the annealing path commonly used in MCMC literature, which is given by the linear interpolation in the space of energies. Inspired by Sequential Monte Carlo, we alternate between deterministic time steps from the TT representation of the flow field and stochastic steps, which include Langevin and resampling steps. These adjust the relative weights of the different modes of the target distribution and anneal to the correct path distribution. We showcase the efficiency of our method on multiple numerical examples.
Figures
Reference graph
Works this paper leans on
-
[19]
B. Máté and F. Fleuret. Learning interpolations between boltzmann densities.Transactions on Machine Learning Research, 2023
work page 2023
-
[1]
M. Albergo, G. Kanwar, and P. Shanahan. Flow-based generative models for Markov chain Monte Carlo in lattice field theory.Physical Review D, 100(3), Aug. 2019
work page 2019
-
[2]
Ambrosio, N
L. Ambrosio, N. Gigli, and G. Savaré.Gradient flows: in metric spaces and in the space of probability measures. Springer Science & Business Media, 2008
2008
- [3]
-
[4]
A. Bandeira, A. Maillard, R. Nickl, and S. Wang. On free energy barriers in gaussian priors and failure of cold start MCMC for high-dimensional unimodal distributions.Philosophical transactions. Series A, Mathematical, physical, and engineering sciences, 381:20220150, 03 2023
work page 2023
- [5]
-
[6]
J. Chemseddine, C. Wald, R. Duong, and G. Steidl. Neural sampling from Boltzmann densities: Fisher-Rao curves in the Wasserstein geometry.arXiv preprint arXiv:2410.03282, 2024
arXiv 2024
-
[7]
P.-Y. Chuang and L. A. Barba. Experience report of physics-informed neural networks in fluid simulations: pitfalls and frustration.arXiv preprint arXiv:2205.14249, 2022
arXiv 2022
Show all 38 references
-
[8]
A. S. Dalalyan and A. G. Karagulyan. User-friendly guarantees for the Langevin Monte Carlo with inaccurate gradient.ArXiv, abs/1710.00095, 2017. 10
2017 arXiv
-
[9]
Del Debbio, J
L. Del Debbio, J. Marsh Rossney, and M. Wilson. Efficient modeling of trivializing maps for lattice ϕ4 theory using normalizing flows: A first look at scalability.Physical Review D, 104(9), Nov. 2021
2021
-
[10]
Del Moral, A
P. Del Moral, A. Doucet, and A. Jasra. Sequential Monte Carlo samplers.Journal of the Royal Statistical Society Series B: Statist. Method., 68(3):411–436, 2006
2006
-
[11]
Dolgov, D
S. Dolgov, D. Kalise, and K. K. Kunisch. Tensor decomposition methods for high-dimensional Hamilton–Jacobi–Bellman equations. SIAM Journal on Scientific Computing, 43(3):A1625– A1650, 2021
2021
-
[12]
Doucet, S
A. Doucet, S. Godsill, and C. Andrieu. On sequential monte carlo sampling methods for bayesian filtering.Statistics and Computing, 10:197–208, 2000
2000
-
[13]
Eigel, J
M. Eigel, J. Neumann, R. Schneider, and S. Wolf. Non-intrusive tensor reconstruction for high-dimensional random pdes.Computational Methods in Applied Mathematics, 19(1):39–53, 2019
2019
-
[14]
Eigel, R
M. Eigel, R. Schneider, and D. Sommer. Dynamical low-rank approximations of solutions to the Hamilton–Jacobi–Bellman equation.Numerical Linear Algebra with Applications, 30(3):e2463, 2023
2023
-
[15]
Hagemann, J
P. Hagemann, J. Hertrich, and G. Steidl. Stochastic normalizing flows for inverse problems: A Markov chains viewpoint.SIAM/ASA Journal on Uncertainty Quantification, 10(3):1162–1190, 2022
2022
-
[16]
Hertrich and R
J. Hertrich and R. Gruhlke. Importance corrected neural JKO sampling. arXiv preprint arXiv:2407.20444, 2024
2024 arXiv
-
[17]
Thealternatinglinearschemefortensoroptimization in the tensor train format.SIAM Journal on Scientific Computing, 34(2):A683–A713, 2012
S.Holtz, T.Rohwedder, andR.Schneider. Thealternatinglinearschemefortensoroptimization in the tensor train format.SIAM Journal on Scientific Computing, 34(2):A683–A713, 2012
2012
-
[18]
Holtz, T
S. Holtz, T. Rohwedder, and R. Schneider. On manifolds of tensors of fixed TT-rank.Nu- merische Mathematik, 120(4):701–731, 2012
2012
-
[20]
A. G. D. G. Matthews, M. Arbel, D. J. Rezende, and A. Doucet. Continual repeated annealed flow transport Monte Carlo. InProceedings of the 39th Int. Conference on Machine Learning, Proceedings of Machine Learning Research, Jul 2022
2022
-
[21]
L. I. Midgley, V. Stimper, G. N. C. Simm, B. Schölkopf, and J. M. Hernández-Lobato. Flow annealed importance sampling bootstrap.arXiv preprint arXiv:2208.01893, 2023
2023 arXiv
-
[22]
Miranda, J
C. Miranda, J. Schütte, D. Sommer, and M. Eigel. Approximating Langevin Monte Carlo with ResNet-like neural network architectures.arXiv preprint arXiv:2311.03242, 2024
2024 arXiv
-
[23]
R. M. Neal. Annealed importance sampling.Statist. and Comp., 11:125–139, 2001. 11
2001
-
[24]
Nouy.Chapter 4: Low-Rank Methods for High-Dimensional Approximation and Model Order Reduction, page 171–226
A. Nouy.Chapter 4: Low-Rank Methods for High-Dimensional Approximation and Model Order Reduction, page 171–226. Society for Industrial and Applied Mathematics, July 2017
2017
-
[25]
F. Noé, S. Olsson, J. Köhler, and H. Wu. Boltzmann generators: Sampling equilibrium states of many-body systems with deep learning.Science, 365(6457), 2019
2019
-
[26]
I. V. Oseledets. Tensor-train decomposition. SIAM Journal on Scientific Computing , 33(5):2295–2317, 2011
2011
-
[27]
I. V. Oseledets. Constructive representation of functions in low-rank tensor formats.Construc- tive Approximation, 37:1–18, 2013
2013
-
[28]
Papamakarios, E
G. Papamakarios, E. Nalisnick, D. J. Rezende, S. Mohamed, and B. Lakshminarayanan. Nor- malizing flows for probabilistic modeling and inference.Journal of Machine Learning Research, 22(57):1–64, 2021
2021
-
[29]
Raissi, P
M. Raissi, P. Perdikaris, and G. Karniadakis. Physics-informed neural networks: A deep learn- ing framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Comp. Phys., 378:686–707, 2019
2019
-
[30]
Richter, L
L. Richter, L. Sallandt, and N. Nüsken. Solving high-dimensional parabolic PDEs using the ten- sor train format. InInternational Conference on Machine Learning, pages 8998–9009. PMLR, 2021
2021
-
[31]
Rohwedder and A
T. Rohwedder and A. Uschmajew. On local convergence of alternating schemes for optimiza- tion of convex problems in the tensor train format.SIAM Journal on Numerical Analysis, 51(2):1134–1162, 2013
2013
-
[32]
Sejdinovic, B
D. Sejdinovic, B. Sriperumbudur, A. Gretton, and K. Fukumizu. Equivalence of distance-based and RKHS-based statistics in hypothesis testing.The Annals of Statistics, 41(5):2263 – 2291, 2013
2013
-
[33]
Sommer, R
D. Sommer, R. Gruhlke, M. Kirstein, M. Eigel, and C. Schillings. Generative modelling with tensor train approximations of Hamilton–Jacobi–Bellman equations. arXiv preprint arXiv:2402.15285, 2024
2024 arXiv
-
[34]
J. Sun, J. Berner, L. Richter, M. Zeinhofer, J. Müller, K. Azizzadenesheli, and A. Anand- kumar. Dynamical measure transport and neural PDE solvers for sampling.arXiv preprint arXiv:2407.07873, 2024
2024 arXiv
-
[35]
G. J. Székely and M. L. Rizzo. Energy statistics: A class of statistics based on distances. Journal of Statistical Planning and Inference, 143(8):1249–1272, 2013
2013
-
[36]
S. K. Vemuri, T. Büchner, J. Niebling, and J. Denzler. Functional tensor decompositions for physics-informed neural networks.arXiv preprint arXiv:2408.13101, 2024
2024 arXiv
-
[37]
H. Wu, J. Köhler, and F. Noé. Stochastic normalizing flows.Advances in Neural Information Processing Systems, 33:5933–5944, 2020
2020
-
[38]
Zhang, Ö
Y. Zhang, Ö. D. Akyildiz, T. Damoulas, and S. Sabanis. Nonasymptotic estimates for stochas- tic gradient Langevin dynamics under local conditions in nonconvex optimization. Applied Mathematics & Optimization, 87(2):25, 2023. 12
2023
Reviewed August 11, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.