REVIEW 1 major objections 5 minor 34 references
On approximating the $f$-divergence between two Ising models
T0 review · 1 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash
Pith's one-line read For zero-field uniform-interaction Ising models, approximating the \(\chi^\alpha\)-divergence has a sharp tractability boundary: an FPRAS exists exactly when \((\beta_\nu/\beta_\mu)^\alpha\beta_\mu \ge (\Delta-2)/\Delta\), and below it…
desk verdict The χ^α-divergence phase-transition result is real and worth refereeing, but the paper's advertised Rényi-divergence extension is mislabeled and the Lemma 2.3 minimizer claim is wrong. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The machinery that carries the argument is the interpolated family of Ising models \(\mathcal F(\nu,\mu,\$\alpha$)\), whose \(k\)-th member has interaction \($J^{{(k)}}$=kJ^\nu-(k-1)J^\mu\) and external field \($h^{{(k)}}$=kh^\nu-(k-1)h^\mu\). Because the weight of a configuration in that model is proportional to \(\nu^k(\$\sigma$)/\$mu^{{k-1}}$(\$\sigma$)\), a binomial expansion of the likelihood ratio rewrites the \(\chi^\$\alpha$\)-divergence as an alternating sum of partition-function ratios \(Z_\$mu^{{k-1}}$Z_k/Z_\nu^k\), where \(Z_k\) is the partition function of the \(k\)-th interpolated model. The algorithm estimates each ratio with counting oracles and samples the interpolated models to decide which side of the likelihood ratio each configuration lies on; an error-control lemma lower-bounds the divergence by \(\sum_\$\sigma$ \mu(\$\sigma$)(\nu(\$\sigma$)/\mu(\$\sigma$)+1)^\$\alpha$\) in the large-parameter-distance case. For small parameter distance, the ratio \(W=w_\nu(X)/w_\mu(X)\) under \(X\sim\mu\) is concentrated and a sample-average estimator works. The hardness direction maps the divergence to the partition function of an anti-ferromagnetic Ising model at inverse temperature \(\$\beta$=(\beta_\nu/\beta_\mu)^\$\alpha$\beta_\mu\), where known inapproximability results for counting apply.
What would settle it
Enumerate all zero-field 3-regular graphs on up to, say, 12 vertices, take \(\$\alpha$=2\), \(\beta_\nu=1/3\), \(\beta_\mu=2/3\), and compute the exact \(\$chi^{2}$\)-divergence and partition functions; Lemma 7.2 predicts the ratio \(D_{\$chi^{2}$}(\nu\|\mu)Z(G,\beta_\nu)^2/Z(G,\beta_\mu)\) stays within constant factors of \(Z(G,1/6)\), so a graph violating those bounds would refute the hardness reduction's core comparison.
Extended reading notes
Core claim
The central claim is a matching algorithm-hardness pair. On the algorithmic side (Theorem 1.4), for any constant integer \(\$\alpha$\ge 1\), if both input Ising models are \(b\)-marginally bounded and every model in the family \(\mathcal F(\nu,\mu,\$\alpha$)=\{(G,$J^{{(k)}}$,$h^{{(k)}}$): $J^{{(k)}}$=kJ^\nu-(k-1)J^\mu,\ $h^{{(k)}}$=kh^\nu-(k-1)h^\mu,\ 0\le k\le \$\alpha$\}\) admits sampling and approximate-counting oracles, then the \(\chi^\$\alpha$\)-divergence can be approximated in polynomial time with relative error \($e^{{\pm\varepsilon}}$\). On the hardness side (Theorem 1.6), fix \(\$\alpha$\ge 2\), \(\$\Delta$\ge 3\), and zero-field uniform-interaction models \((G,\beta_\nu)\), \((G,\beta_\mu)\) with \(\beta_\mu>\beta_\nu\ge(\$\Delta$-2)/\$\Delta$\) and \((\beta_\nu/\beta_\mu)^\$\alpha$\beta_\mu<(\$\Delta$-2)/\$\Delta$\); then, unless NP=RP, no FPRAS exists, even though both input models are individually tractable. For this family the two theorems yield an exact phase transition at \((\beta_\nu/\beta_\mu)^\$\alpha$\beta_\mu=(\$\Delta$-2)/\$\Delta$\). The paper further claims the same oracle-based framework approximates several other standard divergences, including KL and Rényi divergences, requiring oracles only for the input models in those cases.
Load-bearing premise
The load-bearing premise is that every conditional single-spin marginal of both input distributions is bounded below by a fixed positive constant \(b\); if a marginal can be exponentially small, the divergence lower bounds and the algorithm's error control stop working.
Editorial extensions
If this is right
- When the oracle assumptions hold and the \(b\)-marginal bound is constant, Theorem 1.4 gives a fully polynomial randomized scheme for the \(\chi^\alpha\)-divergence, generalizing the TV-distance FPRAS that corresponds to \(\alpha=1\).
- In the zero-field uniform-interaction setting, the paper's two theorems imply an exact phase transition: FPRAS exists if and only if \((\beta_\nu/\beta_\mu)^\alpha\beta_\mu\ge(\Delta-2)/\Delta\), with NP=RP-hardness otherwise.
- Because the hardness instances have both input models individually tractable, the intermediate interpolated model at \(k=\alpha\) is what makes the divergence hard; the oracle assumptions in the algorithmic theorem are therefore necessary, not an artifact.
- For KL, Rényi, and Jensen-Shannon divergences, only the two input models need sampling and counting oracles, so the tractability region is larger than for \(\chi^\alpha\)-divergence.
- When the parameter distance between the two models is small (\(<1/(10(n+3m))\)), any \(f\)-divergence satisfying a mild derivative condition is approximable with just a sampling oracle, no counting oracle.
Reading between the lines
- The threshold expression \((\beta_\nu/\beta_\mu)^\alpha\beta_\mu\) predicts that hardness appears precisely when the interpolated model at \(k=\alpha\) crosses the anti-ferromagnetic uniqueness boundary; if approximate counting for anti-ferromagnetic models were ever extended below that boundary, the divergence phase transition would shift accordingly.
- A natural testable extension is to ask whether the \(b\)-marginal lower bound can be replaced by high-temperature or correlation-decay assumptions; the paper's Lemmas 2.2 and 2.3 suggest the same structure would survive as long as single-site marginals stay bounded away from zero on relevant subsets.
- For real, non-integer \(\alpha\), the binomial expansion breaks, but the phase-transition formula suggests an interpolation-based estimator might still work; checking numerically whether the same threshold governs tractability for, say, \(\alpha=1.5\) would be a concrete next step.
- In practical terms, the small-parameter-distance estimator \(\hat D = \frac1T\sum_i f(W_i/\bar W)\) with \(T=\mathrm{poly}(n/\varepsilon)\) gives an off-the-shelf plug-in recipe whenever likelihood ratios are concentrated; one could benchmark it on lattice Ising models to see whether the worst-case sample count is pessimistic.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper studies the problem of approximating the f-divergence between two Ising models given by their interaction matrices and external fields, up to relative error. For the χ^α-divergence with constant integer α, it gives an FPRAS (Theorem 1.4) under a b-marginal lower bound on both input distributions, assuming sampling and approximate counting oracles for every model in the interpolating family F(ν,μ,α). For zero-field, unified-value Ising models with β_μ>β_ν≥(Δ-2)/Δ, it proves a matching hardness result (Theorem 1.6) when (β_ν/β_μ)^α β_μ < (Δ-2)/Δ, yielding a phase transition. The paper also claims algorithms for KL, Rényi, Jensen-Shannon, α-divergence, and squared Hellinger distance.
Significance. The χ^α phase transition is a substantial, clean result: it provides a complete threshold in a natural constant-parameter regime, extends the earlier TV-distance work of [FLY25] to a general family of divergences, and the hardness direction rests on established anti-ferromagnetic partition-function inapproximability. The proofs are detailed, with explicit running times and oracle assumptions. The b-marginal lower bound is stated clearly and is satisfied in the constant-parameter instances where the phase transition is formulated. The main weakness is the incorrect identification of Rényi divergence in the extension sections, which overstates the paper's contributions even though it does not affect the χ^α results.
major comments (1)
- [Section 1.2, Theorem 1.7, Section 6] The paper's claimed extension to Rényi divergence is mathematically incorrect. The function f(x)=-ln x+x-1, which the paper labels as generating the Rényi divergence, gives D_f(ν||μ)=E_μ[-ln(ν/μ)+ν/μ-1]=KL(μ||ν), the reverse KL divergence. The actual Rényi divergence of order α is (1/(α-1))ln E_μ[(dν/dμ)^α], which is not an f-divergence of the form studied. Consequently Theorem 1.7 and the abstract's assertion that the algorithm extends to Rényi divergence are unsupported; the algorithm in Section 6 estimates reverse KL. The same paragraph in Section 1.2 also misstates the limits of the α-divergence: for f(x)=(x^α-αx-(1-α))/(α(α-1)), the limit α→1 is x ln x-x+1 (KL), and α→0 is -ln x+x-1 (reverse KL), not the reverse of what is written. These statements must be corrected or the Rényi claims removed.
minor comments (5)
- [Lemma 2.3] The proof claims that g(t) attains its minimum at t=(b^{2α}θ^α/2)^{1/(α+1)}, but the derivative g'(t) vanishes at t=(2b^{2α}θ^α)^{1/(α+1)}. The bound remains valid with the stated t, since the inequality holds for every t, but the 'minimum' claim and the subsequent display should be corrected or the optimal t used.
- [Lemma 5.1] The running-time summary states T=Θ_{α,b}(1/(ε^2 θ^α)), but the algorithm in Section 5.1 defines T as ⌈8·10^4(α+1)/(ε^2 B_{α,b}(θ)^2)⌉, and since B_{α,b}(θ)=Θ(θ^α), this is T=Θ_{α,b}(1/(ε^2 θ^{2α})), matching Theorem 1.4. The exponent in Lemma 5.1 should be corrected.
- [Lemma 5.2 proof] In the sentence describing contributions to D_{χ^α}, the formula for ν(σ)>μ(σ) is written as μ(σ)(μ(σ)/ν(σ)-1)^α; it should be μ(σ)(ν(σ)/μ(σ)-1)^α. The following line uses the correct expression.
- [Section 6.1 table] The row labeled 'Rényi' lists f(x)=-ln x+x-1; this is reverse KL, and the table's computed f'' and F(ζ) are correct for that function. The label should be changed to avoid propagating the error described in the major comment.
- [Abstract and Introduction] The abstract and Problem 1.1 state the problem for general Ising models without mentioning the b-marginal lower bound, while Theorem 1.4 requires it. The scope should be qualified, since the b-marginal condition can fail for low-temperature Ising models even when sampling and counting are tractable.
Circularity Check
No circularity: the χ^α results reduce the divergence to partition-function oracles via a genuine interpolation identity; the only flagged issue is a non-circular mislabeling of reverse KL as Rényi divergence.
full rationale
The central derivation chain for χ^α-divergence is self-contained: Eq. (4)-(5) expand D_{χ^α}(ν∥μ) = (1/2)∑_σ μ(σ)|ν(σ)/μ(σ)-1|^α as a fixed binomial combination of the partition functions Z_k of the interpolated family F(ν,μ,α), namely Z_μ^{k-1}Z_k/Z_ν^k, and the algorithm estimates these quantities directly with the assumed sampling/counting oracles. No parameter is fitted to data and no quantity called a prediction is defined in terms of the target divergence, so the core algorithm is not circular by construction. The lower bounds in Lemma 2.2 and Lemma 2.3 are proved in the paper from Lemma 3.1, which is cited from the authors' earlier [FLY25] but is a published, independently verifiable lemma with a proof, not an unverified self-citation; Lemma 4.4 is likewise an external published lemma from [FLY25]. The hardness proof is a genuine reduction from the external antiferromagnetic-partition-function hardness of [SS14, GŠV16] via Lemma 7.2, and the phase-transition threshold β=(βν/βμ)^α βμ is forced by the interpolation family rather than by a fitted constant. The b-marginal assumption is explicit and is verified to hold with constant b on the hardness instances. Separately, the advertised extension to 'Rényi divergence' (Section 1.2, Theorem 1.7) contains a non-circular correctness error: with f(x)=-ln x+x-1, D_f(ν∥μ)=E_μ[ln(μ/ν)]=KL(μ∥ν), the reverse KL divergence, not the standard Rényi divergence; this does not affect the validity of Theorems 1.4 and 1.6.
Assumptions & free parameters
assumptions (6)
- domain assumption Function f satisfies Assumption 1: convex, f(1)=0, f'(x)<0 for 0<x<1, f'(x)>0 for x>1, and f''(x)≥0 off x=1.
- domain assumption Both input Ising models satisfy the b-marginal lower bound (Definition 1.3) for a constant b>0.
- domain assumption The relevant Ising models admit sampling and approximate counting oracles (Definition 1.2) with stated cost functions.
- standard math NP is not equal to RP, and approximating the partition function of an antiferromagnetic Ising model with constant β < (Δ-2)/Δ on Δ-regular graphs is hard (Lemma 7.1, citing [SS14, GŠV16]).
- standard math Polynomial-time sampling and approximate counting exist for ferromagnetic Ising models (β≥1) by [JS93] and for uniqueness-regime models by [CCYZ25].
- standard math Standard mathematical tools are available: Chebyshev, Jensen, mean value theorem, and the data processing inequality for f-divergences.
Cite this review
Pith. "Pith review of On approximating the $f$-divergence between two Ising models." pith.science (2026). https://pith.science/paper/KQB7RFLF
@misc{pith2026250905016,
author = {Pith},
title = {Pith review of: On approximating the $f$-divergence between two Ising models},
year = {2026},
howpublished = {\url{https://pith.science/paper/KQB7RFLF}},
note = {Machine review of arXiv:2509.05016}
}
abstract
The $f$-divergence is a fundamental notion that measures the difference between two distributions. In this paper, we study the problem of approximating the $f$-divergence between two Ising models, which is a generalization of recent work on approximating the TV-distance. Given two Ising models $\nu$ and $\mu$, which are specified by their interaction matrices and external fields, the problem is to approximate the $f$-divergence $D_f(\nu\,\|\,\mu)$ within an arbitrary relative error $\mathrm{e}^{\pm \varepsilon}$. For $\chi^\alpha$-divergence with a constant integer $\alpha$, we establish both algorithmic and hardness results. The algorithm works in a parameter regime that matches the hardness result. Our algorithm can be extended to other $f$-divergences such as $\alpha$-divergence, Kullback-Leibler divergence, R\'enyi divergence, Jensen-Shannon divergence, and squared Hellinger distance.
Reference graph
Works this paper leans on
-
[1]
A circus of circuits: Connections between decision diagrams, circuits, and automata
Antoine Amarilli, Marcelo Arenas, YooJung Choi, Mika \" e l Monet, Guy Van den Broeck, and Benjie Wang. A circus of circuits: Connections between decision diagrams, circuits, and automata. arXiv preprint arXiv:2404.09674 , 2024
arXiv 2024
-
[2]
Sketching, embedding and dimensionality reduction in information theoretic spaces
Amirali Abdullah, Ravi Kumar, Andrew McGregor, Sergei Vassilvitskii, and Suresh Venkatasubramanian. Sketching, embedding and dimensionality reduction in information theoretic spaces. In AISTATS , volume 51, pages 948--956. JMLR.org, 2016
work page 2016
-
[3]
Complexity of high-dimensional identity testing with coordinate conditional sampling
Antonio Blanca, Zongchen Chen, Daniel Stefankovic, and Eric Vigoda. Complexity of high-dimensional identity testing with coordinate conditional sampling. ACM Trans. Algorithms , 21(1):7:1--7:58, 2025
work page 2025
-
[4]
Approximating the total variation distance between G aussians
Arnab Bhattacharyya, Weiming Feng, and Piyush Srivastava. Approximating the total variation distance between G aussians. In AISTATS , volume 258 of Proceedings of Machine Learning Research , pages 1846--1854. PMLR , 2025
work page 2025
-
[5]
Meel, Dimitrios Myrisiotis, A
Arnab Bhattacharyya, Sutanu Gayen, Kuldeep S. Meel, Dimitrios Myrisiotis, A. Pavan, and N. V. Vinodchandran. On approximating total variation distance. In IJCAI , pages 3479--3487. ijcai.org, 2023
2023
-
[6]
Meel, Dimitrios Myrisiotis, A
Arnab Bhattacharyya, Sutanu Gayen, Kuldeep S. Meel, Dimitrios Myrisiotis, A. Pavan, and N. V. Vinodchandran. Total variation distance meets probabilistic inference. In ICML . OpenReview.net, 2024
2024
-
[7]
Meel, Dimitrios Myrisiotis, A
Arnab Bhattacharyya, Sutanu Gayen, Kuldeep S. Meel, Dimitrios Myrisiotis, A. Pavan, and N. V. Vinodchandran. Computational explorations of total variation distance. In ICLR . OpenReview.net, 2025
2025
-
[8]
Algorithms and hardness for estimating statistical similarity
Arnab Bhattacharyya, Sutanu Gayen, Kuldeep S Meel, Dimitrios Myrisiotis, A Pavan, and NV Vinodchandran. Algorithms and hardness for estimating statistical similarity. arXiv preprint arXiv:2502.10527 , 2025
arXiv 2025
Show all 34 references
-
[9]
Meel, and N
Arnab Bhattacharyya, Sutanu Gayen, Kuldeep S. Meel, and N. V. Vinodchandran. Efficient distance approximation for structured high-dimensional distributions via learning. In NeurIPS , 2020
2020
-
[10]
Efficiently learning I sing models on arbitrary graphs
Guy Bresler. Efficiently learning I sing models on arbitrary graphs. In STOC , pages 771--782. ACM , 2015
2015
-
[11]
Cl \' e ment L. Canonne. A Survey on Distribution Testing: Your Data is Big. But is it Blue? Number 9 in Graduate Surveys. Theory of Computing Library, 2020
2020
-
[12]
Cl \' e ment L. Canonne. Topics and techniques in distribution testing: A biased but representative sample. Found. Trends Commun. Inf. Theory , 19(6):1032--1198, 2022
2022
-
[13]
Rapid mixing at the uniqueness threshold
Xiaoyu Chen, Zongchen Chen, Yitong Yin, and Xinyuan Zhang. Rapid mixing at the uniqueness threshold. In STOC , pages 879--890. ACM , 2025
2025
-
[14]
Optimal mixing of G lauber dynamics: E ntropy factorization via high-dimensional expansion
Zongchen Chen, Kuikui Liu, and Eric Vigoda. Optimal mixing of G lauber dynamics: E ntropy factorization via high-dimensional expansion. In STOC , pages 1537--1550. ACM , 2021
2021
-
[15]
Testing I sing models
Constantinos Daskalakis, Nishanth Dikkala, and Gautam Kamath. Testing I sing models. In SODA , pages 1989--2007. SIAM , 2018
1989
-
[16]
A simple polynomial-time approximation algorithm for the total variation distance between two product distributions
Weiming Feng, Heng Guo, Mark Jerrum, and Jiaheng Wang. A simple polynomial-time approximation algorithm for the total variation distance between two product distributions. TheoretiCS , 2, 2023
2023
-
[17]
On deterministically approximating total variation distance
Weiming Feng, Liqiang Liu, and Tianren Liu. On deterministically approximating total variation distance. In SODA , pages 1766--1791. SIAM , 2024
2024
-
[18]
Approximating the total variation distance between spin systems
Weiming Feng, Hongyang Liu, and Minji Yang. Approximating the total variation distance between spin systems. In COLT , volume 291 of Proceedings of Machine Learning Research , pages 1974--2025. PMLR , 2025
1974
-
[19]
Sampling and identity-testing without approximate tensorization of entropy
William Gay, William He, Nicholas Kocurek, and Ryan O'Donnell. Sampling and identity-testing without approximate tensorization of entropy. arXiv preprint arXiv:2506.23456 , 2025
2025 arXiv
-
[20]
Inapproximability of the partition function for the antiferromagnetic I sing and hard-core models
Andreas Galanis, Daniel S tefankovi c , and Eric Vigoda. Inapproximability of the partition function for the antiferromagnetic I sing and hard-core models. Combin. Probab. Comput. , 25(4):500--559, 2016
2016
-
[21]
Polynomial-time approximation algorithms for the I sing model
Mark Jerrum and Alistair Sinclair. Polynomial-time approximation algorithms for the I sing model. SIAM J. Comput. , 22(5):1087--1116, 1993
1993
-
[22]
Sharp bounds on aggregate expert error
Aryeh Kontorovich and Ariel Avital. Sharp bounds on aggregate expert error. In ALT , volume 272 of Proceedings of Machine Learning Research , pages 653--663. PMLR , 2025
2025
-
[23]
On computing the total variation distance of hidden M arkov models
Stefan Kiefer. On computing the total variation distance of hidden M arkov models. In ICALP , volume 107 of LIPIcs , pages 130:1--130:13. Schloss Dagstuhl - Leibniz-Zentrum f \" u r Informatik, 2018
2018
-
[24]
On the tensorization of the variational distance
Aryeh Kontorovich. On the tensorization of the variational distance. Electron. Commun. Probab. , 30:Paper No. 32, 10, 2025
2025
-
[25]
f -divergence estimation and two-sample homogeneity test under semiparametric density-ratio models
Takafumi Kanamori, Taiji Suzuki, and Masashi Sugiyama. f -divergence estimation and two-sample homogeneity test under semiparametric density-ratio models. IEEE Trans. Inf. Theory , 58(2):708--720, 2012
2012
-
[26]
Correlation decay up to uniqueness in spin systems
Liang Li, Pinyan Lu, and Yitong Yin. Correlation decay up to uniqueness in spin systems. In SODA , pages 67--84. SIAM , 2013
2013
-
[27]
Markov chains and mixing times , volume 107
David A Levin and Yuval Peres. Markov chains and mixing times , volume 107. American Mathematical Soc., 2017
2017
-
[28]
Loong Kuan Lee, Nico Piatkowski, Fran c ois Petitjean, and Geoffrey I. Webb. Computing divergences between discrete decomposable models. In AAAI , pages 12243--12251. AAAI Press, 2023
2023
-
[29]
The I sing partition function: Zeros and deterministic approximation
Jingcheng Liu, Alistair Sinclair, and Piyush Srivastava. The I sing partition function: Zeros and deterministic approximation. Journal of Statistical Physics , 174(2):287--315, 2019
2019
-
[30]
Schneider
Barnab \' a s P \' o czos and Jeff G. Schneider. On the estimation of -divergences. In AISTATS , volume 15 of JMLR Proceedings , pages 609--617. JMLR.org, 2011
2011
-
[31]
Rubenstein, Olivier Bousquet, Josip Djolonga, Carlos Riquelme, and Ilya O
Paul K. Rubenstein, Olivier Bousquet, Josip Djolonga, Carlos Riquelme, and Ilya O. Tolstikhin. Practical and consistent estimation of f -divergences. In NeurIPS , pages 4072--4082, 2019
2019
-
[32]
Neural estimation of statistical divergences
Sreejith Sreekumar and Ziv Goldfeld. Neural estimation of statistical divergences. J. Mach. Learn. Res. , 23:126:1--126:75, 2022
2022
-
[33]
Counting in two-spin models on d -regular graphs
Allan Sly and Nike Sun. Counting in two-spin models on d -regular graphs. Ann. Probab. , 42(6):2383--2416, 2014
2014
-
[34]
A complete problem for statistical zero knowledge
Amit Sahai and Salil Vadhan. A complete problem for statistical zero knowledge. J. ACM , 50(2):196--249, 2003
2003
Reviewed August 15, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.