Pith. sign in

REVIEW 33 references

Integrating Neural Networks and Tensor Networks for Computing Free Energy

T0 review · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Tensor-network-based variational autoregressive networks (TNVAN) fix a small 'width set' of spins, contract the rest of the network exactly, and fit a neural variational distribution on the reduced system to estimate free energy.

arxiv 2504.12037 v1 pith:TECMMJHC submitted 2025-04-16 cond-mat.stat-mech

Hanyan Cao , Yijia Wang , Feng Pan , Pan Zhang This is my paper · ORCID
classification cond-mat.stat-mech
keywords networksmethodsystemstensorneuralvariationalenergyfree
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Computing the free energy of a spin system requires summing over an astronomically large number of configurations. Tensor networks compute such sums accurately in low dimensions, but become too expensive in dense or high-dimensional systems. Variational neural networks handle arbitrary graphs, but their estimates are often less precise. This paper combines the two. The method first chooses a small set of spins called the width set. Once those spins are fixed, the remaining network is contracted exactly, which turns its influence into extra interactions among the width set spins. The problem is now a small, dense effective model. An autoregressive neural network samples from this reduced model and minimizes a variational free energy, which is guaranteed to be an upper bound on the true free energy because of the non-negativity of KL divergence. The numerical section compares TNVAN with variational autoregressive networks, convolutional variants, and a previous feedback vertex set method. On a 40 by 40 Ising model near the critical temperature, the reported relative error is around 10^-4, versus 10^-1 and 10^-2 for the two neural baselines. On random regular graphs and on the fully connected Sherrington-Kirkpatrick model, TNVAN reaches lower free energy upper bounds than the baselines. The main caveats are that no code, data, or error bars are provided, and for the random graph and SK cases no exact reference free energy is shown, so absolute accuracy is not directly verified.
Extended reading notes

Core claim

TNVAN 'consistently achieves lower variational free energy' and 'superior accuracy' on 2D Ising, random regular graph spin glasses, and the SK model. In the 40 by 40 Ising test near Tc, 'TNVAN (with |W|=33) achieves a relative error of the order of 10^-4, compared to 10^-1 for VAN and 10^-2 for Conv-VAN.' If the paper is correct, projecting the system onto a bounded-width tensor network before training a variational autoregressive network yields materially tighter free energy upper bounds than purely neural variational baselines.

Load-bearing premise

The method's upper-bound property depends on the remaining tensor network being contracted exactly for every fixed configuration of the width set. The paper states that TNVAN avoids SVD and uses 'batch-contraction to contract multiple tensor networks', but it does not show that the contracted effective energy Etilde(s) is exact for all reported wu values on random graphs and the SK model. If any approximate contraction or untracked normalization error enters, Fq = E_q[Etilde + (1/beta) ln q] is no longer a valid upper bound on the true free energy. A second load-bearing premise is that the width set found by Cotengra or simulated annealing is small enough for exact contraction, which is heuristic rather than guaranteed.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The paper introduces no new physical entities. The width set is a computational subset of spins, not a new object with independent physical evidence. The central claim rests on exact tensor contraction, valid width set selection, and successful nonconvex neural optimization, all of which are stated but only partially verified in the numerical section.

free parameters (2)
  • width upper bound wu = wu = {2, 4, 10, 20} for 2D Ising; wu = 20 or {1, 3, 8, 13, 17, 20} for random graphs; wu = 20 for SK
    Chosen by hand as a computational budget. It controls the width set size and the accuracy-cost trade-off, but is not fitted to the target free energy.
  • VAN hyperparameters = depth=1, channels=2, learning rate=0.01, batch size=1000 or 2000, max steps=1000 or 2000
    Hand-selected for the experiments. These affect numerical accuracy and convergence but not the validity of the variational upper bound.
assumptions (4)
  • standard math Non-negativity of KL divergence implies the variational free energy is an upper bound on the true free energy.
    Used in Eq. (5) and throughout Section III. This is a standard convexity fact and is correctly applied.
  • domain assumption The remaining tensor network is contracted exactly, so Etilde(s) equals -(1/beta) ln sum_r exp(-beta E2(s,r)).
    Required for Eq. (4) to be exact and for Fq to remain a valid upper bound. The paper states it avoids SVD and uses exact contraction, but does not verify exactness on the reported random graph and SK runs.
  • domain assumption The width set and contraction order found by Cotengra and simulated annealing are valid and computationally feasible.
    Used in Algorithm 1 step 1. Finding an optimal width set and contraction order is NP-hard, so the heuristic output is assumed adequate.
  • domain assumption The autoregressive network is expressive enough and the nonconvex optimization converges to a good optimum.
    The reported accuracy of Fq depends on the variational optimization succeeding. No convergence guarantees are provided.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Integrating Neural Networks and Tensor Networks for Computing Free Energy." pith.science (2026). https://pith.science/paper/TECMMJHC

@misc{pith2026250412037,
  author       = {Pith},
  title        = {Pith review of: Integrating Neural Networks and Tensor Networks for Computing Free Energy},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TECMMJHC}},
  note         = {Machine review of arXiv:2504.12037}
}
read the original abstract

Computing free energy is a fundamental problem in statistical physics. Recently, two distinct methods have been developed and have demonstrated remarkable success: the tensor-network-based contraction method and the neural-network-based variational method. Tensor networks are accu?rate, but their application is often limited to low-dimensional systems due to the high computational complexity in high-dimensional systems. The neural network method applies to systems with general topology. However, as a variational method, it is not as accurate as tensor networks. In this work, we propose an integrated approach, tensor-network-based variational autoregressive networks (TNVAN), that leverages the strengths of both tensor networks and neural networks: combining the variational autoregressive neural network's ability to compute an upper bound on free energy and perform unbiased sampling from the variational distribution with the tensor network's power to accurately compute the partition function for small sub-systems, resulting in a robust method for precisely estimating free energy. To evaluate the proposed approach, we conducted numerical experiments on spin glass systems with various topologies, including two-dimensional lattices, fully connected graphs, and random graphs. Our numerical results demonstrate the superior accuracy of our method compared to existing approaches. In particular, it effectively handles systems with long-range interactions and leverages GPU efficiency without requiring singular value decomposition, indicating great potential in tackling statistical mechanics problems and simulating high-dimensional complex systems through both tensor networks and neural networks.

Figures

Figures reproduced from arXiv: 2504.12037 by the authors.

Figure 1
Figure 1. FIG. 1. TNVAN method. (a) The 2D Ising model on a 7 [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. FIG. 2. Free energies of 40 [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. FIG. 3. Free energies of spin glass systems. (a) Size of the width set [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

33 extracted references · 24 canonical work pages

  1. [1]

    Using the existing method to obtain W and the contraction order π of the remaining tensor network

    work page

  2. [2]

    for 0≤i< Epoch do (a) Draw samples from the nerual network based on Eq.8

    Construct a variational neural network. for 0≤i< Epoch do (a) Draw samples from the nerual network based on Eq.8. (b) Compute the effective energy ˜E(s) of these samples using tensor network contraction. (c) Compute the loss function Fq and its gradient based on Eq.9 and Eq.10. (d) Using the above gradient update parameters as well as the loss function. e...

    work page

  3. [3]

    We calculated the free energy at different temperatures using different variational methods and CATN with different hyper-parameters

    (d) Sherrington-Kirkpatrick model with n = 50, and Jij sampled from a standard normal distribution N (0, 1). We calculated the free energy at different temperatures using different variational methods and CATN with different hyper-parameters. Inset is the discrepancy of other variational methods relative to TNVAN. For comparison, the hyper-parameters of a...

    work page 2000

  4. [4]

    Landau and E

    L. Landau and E. Lifshitz, Statistical Physics , Vol. 5 (Elsevier, 2013)

    work page 2013

  5. [5]

    J. P. Sethna, Statistical Mechanics: Entropy, Order Parameters, and Complexity (Oxford University Press, 2021). 9

    work page 2021

  6. [6]

    M. I. Jordan, Z. Ghahramani, T. S. Jaakkola, and L. K. Saul, An introduction to variational methods for graphical models, Machine learning 37, 183 (1999)

    work page 1999

  7. [7]

    H. A. Bethe, Statistical theory of superlattices, Proceedings of the Royal Society of London. Series A-Mathematical and Physical Sciences 150, 552 (1935)

    work page 1935

  8. [8]

    Kikuchi, A theory of cooperative phenomena, Phys

    R. Kikuchi, A theory of cooperative phenomena, Phys. Rev. 81, 988 (1951)

    1951

Show all 33 references
  1. [9]

    D. J. MacKay, Information theory, inference and learning algorithms (Cambridge university press, 2003)

    2003

  2. [10]

    Levin and C

    M. Levin and C. P. Nave, Tensor renormalization group approach to two-dimensional classical lattice models, Physical review letters 99, 120601 (2007)

    2007

  3. [11]

    D. Wu, L. Wang, and P. Zhang, Solving statistical mechanics using variational autoregressive networks, Phys. Rev. Lett. 122, 080602 (2019)

    2019

  4. [12]

    F. Pan, P. Zhou, H.-J. Zhou, and P. Zhang, Solving statistical mechanics on sparse graphs with feedback-set variational autoregressive networks, Phys. Rev. E 103, 012103 (2021)

    2021

  5. [13]

    Germain, K

    M. Germain, K. Gregor, I. Murray, and H. Larochelle, Made: Masked autoencoder for distribution estimation (2015), arXiv:1502.03509 [cs.LG]

    2015 arXiv

  6. [14]

    F. Pan, P. Zhou, S. Li, and P. Zhang, Contracting arbitrary tensor networks: general approximate algorithm and applica- tions in graphical models and quantum circuit simulations, Physical Review Letters 125, 060503 (2020)

    2020

  7. [15]

    Festa, P

    P. Festa, P. M. Pardalos, and M. G. C. Resende, Feedback set problems, in Encyclopedia of optimization (Springer, 2012) pp. 1–13

    2012

  8. [16]

    Gray and S

    J. Gray and S. Kourtis, Hyper-optimized tensor network contraction, Quantum 5, 410 (2021)

    2021

  9. [17]

    J. Chen, F. Zhang, C. Huang, M. Newman, and Y. Shi, Classical simulation of intermediate-size quantum circuits, arXiv preprint arXiv:1805.01450 (2018)

    2018 arXiv

  10. [18]

    Villalonga, S

    B. Villalonga, S. Boixo, B. Nelson, C. Henze, E. Rieffel, R. Biswas, and S. Mandr` a, A flexible high-performance simulator for verifying and benchmarking quantum circuits implemented on real hardware, npj Quantum Information 5, 86 (2019)

    2019

  11. [19]

    Pan and P

    F. Pan and P. Zhang, Simulation of quantum circuits using the big-batch tensor network method, Physical Review Letters 128, 10.1103/PhysRevLett.128.030501 (2022)

    2022 doi

  12. [20]

    F. Pan, K. Chen, and P. Zhang, Solving the sampling problem of the sycamore quantum circuits, Physical Review Letters 129, 090502 (2022)

    2022

  13. [21]

    J. Xu, H. Zhang, L. Liang, L. Deng, Y. Xie, and G. Li, Np-hardness of tensor network contraction ordering (2023), arXiv:2310.06140 [cs.CC]

    2023 arXiv

  14. [22]

    K. Kask, A. E. Gelfand, L. Otten, and R. Dechter, Pushing the power of stochastic greedy ordering schemes for inference in graphical models, in Proceedings of the Twenty-Fifth AAAI Conference on Artificial Intelligence (AAAI-11) (AAAI Press,

  15. [23]

    Kalachev, P

    G. Kalachev, P. Panteleev, and M.-H. Yung, Multi-tensor contraction for xeb verification of quantum circuits, arXiv (2022), 2108.05665

    2022 arXiv

  16. [24]

    Kalchbrenner, A

    N. Kalchbrenner, A. van den Oord, K. Simonyan, I. Danihelka, O. Vinyals, A. Graves, and K. Kavukcuoglu, Video pixel networks, in Proceedings of the 34th International Conference on Machine Learning , Proceedings of Machine Learning Research, Vol. 70, edited by D. Precup and Y....

    2017

  17. [25]

    Vaswani, N

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin, Attention is all you need, Advances in neural information processing systems 30 (2017)

    2017

  18. [26]

    R. S. Sutton, Reinforcement learning: An introduction, A Bradford Book (2018)

    2018

  19. [27]

    A. N. Avramidis and J. R. Wilson, Integrated variance reduction strategies for simulation, Operations Research 44, 327 (1996)

    1996

  20. [28]

    Kac and J

    M. Kac and J. C. Ward, A combinatorial solution of the two-dimensional ising model, Physical Review 88, 1332 (1952)

    1952

  21. [29]

    D. J. Thouless, P. W. Anderson, and R. G. Palmer, Solution of’solvable model of a spin glass’, Philosophical Magazine 35, 593 (1977)

    1977

  22. [30]

    J. S. Yedidia, W. T. Freeman, Y. Weiss,et al., Understanding belief propagation and its generalizations, Exploring artificial intelligence in the new millennium 8, 0018 (2003)

    2003

  23. [31]

    Viana and A

    L. Viana and A. J. Bray, Phase diagrams for dilute spin glasses, Journal of Physics C: Solid State Physics 18, 3037 (1985)

    1985

  24. [32]

    Sherrington and S

    D. Sherrington and S. Kirkpatrick, Solvable model of a spin-glass, Physical review letters 35, 1792 (1975)

    1975

  25. [33]

    F. Pan, P. Zhou, S. Li, and P. Zhang, Contracting arbitrary tensor networks: General approximate algorithm and appli- cations in graphical models and quantum circuit simulations, Phys. Rev. Lett. 125, 060503 (2020)

    2020

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.