Singular Fluctuation as Specific Heat in Bayesian Learning

Sean Plummer

arxiv: 2512.21411 · v3 · submitted 2025-12-24 · 🧮 math.ST · stat.ML· stat.TH

Singular Fluctuation as Specific Heat in Bayesian Learning

Sean Plummer This is my paper

Pith reviewed 2026-05-16 19:20 UTC · model grok-4.3

classification 🧮 math.ST stat.MLstat.TH

keywords singular fluctuationspecific heatBayesian free energytempered posteriorWAICsingular learning theorygeneralization errorRLCT

0 comments

The pith

Singular fluctuation equals the curvature of Bayesian free energy with respect to inverse temperature.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper establishes that under a tempered posterior, singular fluctuation is precisely the second derivative of the Bayesian free energy with respect to the inverse temperature. This quantity also equals the variance of the log-likelihood observable, giving singular fluctuation the role of specific heat in the statistical setting. A reader would care because the identity unifies the geometric meaning of fluctuation with its control over training-versus-generalization error and the complexity term in WAIC. The result uses only existing asymptotic tools from singular learning theory rather than new expansions.

Core claim

Under a tempered (Gibbs) posterior, singular fluctuation is exactly the curvature of the Bayesian free energy with respect to inverse temperature; equivalently, the variance of the log-likelihood observable. In this sense, singular fluctuation is the statistical analogue of specific heat. This clarifies why singular fluctuation governs the equation of state between training and generalization error and why WAIC succeeds on singular models by estimating a fluctuation coefficient rather than a parameter dimension.

What carries the argument

The curvature of the Bayesian free energy with respect to inverse temperature, which equals the variance of the log-likelihood observable under the tempered posterior.

If this is right

Singular fluctuation controls the equation of state relating training and generalization error.
WAIC estimates a fluctuation coefficient rather than a parameter dimension.
As temperature decreases, posterior reorganization suppresses fluctuation directions that affect predictive performance.
Model-specific geometric observables track the decay of singular fluctuation.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

The thermodynamic view suggests estimating singular fluctuation from response functions computed during posterior sampling rather than direct variance estimation.
The same curvature identity may extend to other information criteria that involve second derivatives of free energies in non-identifiable models.
Numerical checks of the identity in finite samples could quantify how quickly the asymptotic equality is approached as data size grows.

Load-bearing premise

The identity is derived within the existing asymptotic framework of singular learning theory and tempered posteriors, inheriting all regularity conditions already required by the RLCT and WAIC literature.

What would settle it

Numerical evaluation in a low-dimensional Gaussian mixture model where the sample variance of the log-likelihood under the tempered posterior fails to equal the finite-difference curvature of the free energy with respect to inverse temperature.

Figures

Figures reproduced from arXiv: 2512.21411 by Sean Plummer.

**Figure 2.** Figure 2: Posterior occupancy distributions for the Gaussian mixture model at representative inverse [PITH_FULL_IMAGE:figures/full_fig_p010_2.png] view at source ↗

**Figure 3.** Figure 3: Thermodynamic response in reduced-rank regression. Left: Posterior variance of the [PITH_FULL_IMAGE:figures/full_fig_p011_3.png] view at source ↗

**Figure 4.** Figure 4: Empirical validation of the equation of state [PITH_FULL_IMAGE:figures/full_fig_p011_4.png] view at source ↗

read the original abstract

Singular learning theory characterizes Bayesian models with non-identifiable parameterizations through two central quantities: the real log canonical threshold (RLCT), which governs marginal likelihood asymptotics, and the singular fluctuation, which determines second-order generalization behavior and the complexity term in WAIC. While the geometric meaning of the RLCT is well understood, the interpretation of singular fluctuation has remained comparatively opaque. We show that singular fluctuation admits a precise thermodynamic interpretation. Under a tempered (Gibbs) posterior, it is exactly the curvature of the Bayesian free energy with respect to inverse temperature; equivalently, the variance of the log-likelihood observable. In this sense, singular fluctuation is the statistical analogue of specific heat. This identity clarifies why singular fluctuation controls the equation of state relating training and generalization error and explains the success of WAIC in singular models: WAIC estimates a fluctuation coefficient rather than a parameter dimension. Across Gaussian mixture models and reduced-rank regression, we demonstrate that singular fluctuation behaves as a thermodynamic response coefficient. As temperature decreases, posterior reorganization suppresses fluctuation directions that affect predictive performance, and model-specific geometric observables track the decay of singular fluctuation. Rather than introducing new asymptotic expansions, this work unifies existing variance identities, equation-of-state results, and WAIC complexity corrections under a single free-energy curvature framework.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

The paper's real move is to identify singular fluctuation with the second derivative of the Bayesian free energy under the tempered posterior, turning it into the specific-heat analogue.

read the letter

The central result is straightforward: under the Gibbs posterior, singular fluctuation equals the curvature of the free energy with respect to inverse temperature, which is also the variance of the log-likelihood observable. This is not a new asymptotic expansion but a clean thermodynamic reading that ties together variance identities already in the singular-learning literature and explains why the fluctuation term appears in both generalization error and the WAIC correction term. The paper shows that as temperature drops, posterior reorganization suppresses the directions that drive fluctuation, and it tracks this decay with model-specific geometric observables in the Gaussian-mixture and reduced-rank regression examples. That unification is the useful part. It makes the existing equation-of-state relations more intuitive without changing the underlying regularity conditions inherited from the RLCT framework. The demonstrations are presented as direct numerical checks of the response-coefficient behavior, though they stay at the level of qualitative tracking rather than quantitative error analysis or extensive controls. Because the work stays inside the standard assumptions of singular learning theory, it does not relax any of those conditions or produce new bounds. The contribution is interpretive rather than technical, which is fine as long as the framing holds up. This is the kind of note that helps people already inside the field see the fluctuation coefficient as a physical response rather than an opaque constant. Readers working on Bayesian asymptotics, WAIC extensions, or geometric learning theory will get the most out of it. I would send it to peer review; the unification is worth referee scrutiny even if the derivations are rearrangements of prior results.

Referee Report

0 major / 4 minor

Summary. The manuscript claims that, under a tempered (Gibbs) posterior, the singular fluctuation of singular learning theory equals the second derivative of the Bayesian free energy with respect to inverse temperature; equivalently, it is the variance of the log-likelihood observable. This supplies a thermodynamic interpretation of singular fluctuation as specific heat, unifies existing variance identities and equation-of-state relations already present in the RLCT/WAIC literature, and explains why WAIC estimates a fluctuation coefficient rather than a parameter dimension. The identity is demonstrated numerically on Gaussian mixture models and reduced-rank regression, where posterior reorganization at low temperature suppresses fluctuation directions that affect predictive performance.

Significance. If the central identity holds, the work supplies a physically transparent account of why singular fluctuation governs the training-generalization relation and why WAIC succeeds for non-identifiable models. Because the argument re-derives no new asymptotic expansions and inherits the regularity conditions already required for the RLCT and WAIC, its contribution is primarily conceptual unification rather than technical extension. The numerical checks on two standard singular model classes provide direct verification of the thermodynamic response coefficient, though they remain qualitative.

minor comments (4)

Abstract: the numerical demonstrations on Gaussian mixture models and reduced-rank regression are mentioned without any quantitative metrics, error bars, or controls; a single sentence summarizing the observed decay rates or agreement with the curvature identity would improve the abstract.
§3 (presumed derivation section): although the identity is stated precisely and linked to prior variance results, the manuscript should include an explicit, self-contained derivation from the definition of the tempered posterior and free energy to the variance expression, even if it follows standard steps; this would make the unification transparent to readers unfamiliar with the earlier literature.
§4: the figures tracking singular fluctuation versus temperature should report statistical variability (e.g., standard errors across independent runs) and indicate the temperature range over which the RLCT/WAIC regularity conditions remain valid.
Notation: the symbol for inverse temperature and the precise definition of the tempered posterior should be restated once in the main text before the central identity, rather than relying solely on references to earlier papers.

Simulated Author's Rebuttal

0 responses · 0 unresolved

We thank the referee for the positive and accurate summary of our manuscript, which correctly identifies the central result as the thermodynamic interpretation of singular fluctuation as specific heat under the tempered posterior. We appreciate the recognition that the work provides conceptual unification of existing variance identities and WAIC results without new asymptotic expansions. The recommendation for minor revision is noted; since no specific major comments were raised in the report, we have no point-by-point responses to address.

Circularity Check

0 steps flagged

No significant circularity; unification of prior identities

full rationale

The paper explicitly states that it unifies existing variance identities, equation-of-state results, and WAIC corrections under a free-energy curvature framework without introducing new asymptotic expansions. The central claim equates singular fluctuation to the second derivative of the Bayesian free energy (or variance of the log-likelihood) within the established singular learning theory setting. This equivalence is derived from the pre-existing regularity conditions of the RLCT/WAIC literature rather than reducing any prediction to a fitted input or self-citation by construction. No load-bearing step exhibits the enumerated circular patterns.

Axiom & Free-Parameter Ledger

0 free parameters · 2 axioms · 0 invented entities

The central claim rests on the standard axioms of singular learning theory (existence of RLCT, tempered posterior construction) and on prior variance identities for the log-likelihood; no new free parameters or invented entities are introduced.

axioms (2)

domain assumption The model belongs to the singular regime where the parameter space is non-identifiable and the RLCT governs asymptotics.
Invoked throughout the abstract as the setting in which singular fluctuation is defined.
domain assumption The tempered (Gibbs) posterior is well-defined and the free-energy curvature exists.
Required for the curvature interpretation to hold.

pith-pipeline@v0.9.0 · 5518 in / 1269 out tokens · 38776 ms · 2026-05-16T19:20:22.390818+00:00 · methodology

discussion (0)

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

Using Statistical Mechanics to Improve Real-World Bayesian Inference: A New Method Combining Tempered Posteriors and Wang-Landau Sampling
stat.ME 2026-04 unverdicted novelty 5.0

Tempered posteriors combined with Wang-Landau sampling identify transition temperatures that optimize predictive performance in Bayesian inference for real-world problems.

Reference graph

Works this paper leans on

1 extracted references · 1 canonical work pages · cited by 1 Pith paper

[1]

Cambridge University Press, 2009

Sumio Watanabe.Algebraic Geometry and Statistical Learning Theory. Cambridge University Press, 2009. Sumio Watanabe. Asymptotic equivalence of bayes cross validation and widely applicable information criterion in singular learning theory.Journal of Machine Learning Research, 11:3571–3594, 2010. Sumio Watanabe. A widely applicable bayesian information crit...

work page 2009

[1] [1]

Cambridge University Press, 2009

Sumio Watanabe.Algebraic Geometry and Statistical Learning Theory. Cambridge University Press, 2009. Sumio Watanabe. Asymptotic equivalence of bayes cross validation and widely applicable information criterion in singular learning theory.Journal of Machine Learning Research, 11:3571–3594, 2010. Sumio Watanabe. A widely applicable bayesian information crit...

work page 2009