Non-Equilibrium Model Selection via Finite-Time Thermodynamics

Masayuki Ohzeki

REVIEW 1 major objections 1 minor 12 references

Non-Equilibrium Model Selection via Finite-Time Thermodynamics

T0 review · 1 major / 1 minor · reviewed 2026-06-27 · grok-4.3

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Finite-time learning dynamics under resource constraints produce a computable analogue of the WBIC for singular models.

desk verdict This paper gives a finite-time WBIC from non-equilibrium dynamics, conditional on an analytic effective potential. read the letter →

arxiv 2606.16399 v2 pith:QHQSKAXB submitted 2026-06-15 cond-mat.dis-nn math.STstat.TH

Masayuki Ohzeki This is my paper · ORCID

classification cond-mat.dis-nnmath.STstat.TH

keywords modelselectionsingularlearningtheoryfinite-timethermodynamicsWBICMDLepiplexitynon-equilibriumdynamics

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper replaces the usual equilibrium posterior in information criteria with an effective ensemble created by learning dynamics that run for finite time under a resource limit. When that ensemble has an analytic effective potential, singular learning theory supplies a resource-dependent real log canonical threshold. The resulting quantity supplies a thermodynamic term usable in time-bounded minimum description length and identifies a finite-time version of singular complexity that tracks structural information captured by epiplexity. A sympathetic reader cares because everyday model training never reaches equilibrium and therefore needs selection rules that explicitly respect time and compute budgets.

What carries the argument

Effective ensemble generated by finite-time learning dynamics under a resource constraint, which yields a resource-dependent real log canonical threshold when an analytic effective potential exists.

What would settle it

Compute the resource-dependent real log canonical threshold for a known singular model and check whether it fails to match the structural complexity revealed by direct epiplexity measurements on the same finite-time trajectories.

Watch

Extended reading notes

Core claim

Replacing the equilibrium posterior with an effective ensemble generated by learning dynamics under a resource constraint, and assuming the ensemble admits an analytic effective potential, allows singular learning theory to yield a resource-dependent real log canonical threshold. The resulting estimator gives a computable thermodynamic contribution to time-bounded MDL and identifies the finite-time singular complexity relevant to the structural information measured by epiplexity.

Load-bearing premise

The effective ensemble created by the learning dynamics admits an analytic effective potential.

Editorial extensions

If this is right

A finite-time analogue of WBIC becomes available for model selection.
Time-bounded MDL acquires an explicit thermodynamic term.
Finite-time singular complexity is identified as a distinct quantity from its equilibrium counterpart.
The threshold depends explicitly on the resource constraint imposed during learning.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Model selection rules could be adjusted on the fly by monitoring resource use rather than waiting for convergence.
Different learning algorithms or schedules would produce different effective complexities even for the same model class.
The approach opens a route to compare singular models trained with deliberately limited compute budgets.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Desk Editor's Note

read the letter

This paper offers a finite-time version of WBIC constructed from non-equilibrium learning dynamics with a resource constraint. When the resulting ensemble has an analytic effective potential, singular learning theory produces a resource-dependent real log canonical threshold.

The new element is the explicit dependence on finite time or resources, which moves the criterion away from the usual equilibrium limit. It positions the estimator as a thermodynamic term in time-bounded MDL and links it to epiplexity as a measure of structural information.

The paper does a clean job of staying within the singular learning theory setup and stating the condition under which the extension works. That keeps the claim precise rather than overreaching. The stress test confirms there is no internal contradiction in the conditional statement.

The main limitation is that the result is conditional on the analytic effective potential. The text states the consequence when the condition holds but does not claim it for arbitrary dynamics or models. Without explicit derivations or validation on concrete examples, it remains unclear how often this applies or how the threshold is computed in practice. The reliance on prior singular learning theory is fine, but the finite-time part needs to stand on its own derivations.

This is for people already working in singular learning theory or non-equilibrium approaches to machine learning. A reader who knows the equilibrium WBIC and RLCT literature will see the extension immediately.

I would send it to peer review. The idea is coherent and the conditional framing is honest, so referees can assess whether the derivations support the claims and how broad the applicability is.

Referee Report

1 major / 1 minor

Summary. The manuscript proposes a finite-time analogue of the Widely Applicable Bayesian Information Criterion (WBIC) for singular learning machines. It replaces the equilibrium posterior with an effective ensemble generated by learning dynamics under a resource constraint. When this ensemble admits an analytic effective potential, singular learning theory is invoked to obtain a resource-dependent real log canonical threshold (RLCT). The resulting estimator is claimed to supply a computable thermodynamic contribution to time-bounded minimum description length (MDL) and to identify the finite-time singular complexity relevant to the structural information measured by epiplexity.

Significance. If the construction is valid and non-circular, the work would extend equilibrium information criteria to non-equilibrium regimes, providing a thermodynamic framing for resource-constrained model selection in singular models. This could link finite-time learning dynamics to complexity measures such as epiplexity, with potential relevance to neural network training under computational constraints. The conditional nature of the claim (analytic effective potential required) limits immediate applicability but, if substantiated with examples, would represent a conceptual advance in bridging statistical mechanics and statistical learning theory.

major comments (1)

The central claim is conditional on the ensemble admitting an analytic effective potential, after which standard SLT supplies the resource-dependent RLCT. The abstract and construction do not assert that such potentials exist for arbitrary dynamics or models; they only state the consequence when the condition holds. No internal contradiction is visible, but the load-bearing step requires explicit conditions or examples where the effective potential is analytic for typical learning dynamics.

minor comments (1)

The provided abstract is entirely high-level and contains no equations, derivations, or validation steps, which prevents assessment of the mathematical support from the given material alone.

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for their insightful comments and the positive evaluation of the potential significance of our work. We provide a point-by-point response to the major comment below.

read point-by-point responses

Referee: The central claim is conditional on the ensemble admitting an analytic effective potential, after which standard SLT supplies the resource-dependent RLCT. The abstract and construction do not assert that such potentials exist for arbitrary dynamics or models; they only state the consequence when the condition holds. No internal contradiction is visible, but the load-bearing step requires explicit conditions or examples where the effective potential is analytic for typical learning dynamics.

Authors: The referee accurately observes that our central claim is conditional upon the effective ensemble admitting an analytic effective potential. This condition is explicitly articulated in the abstract and is central to the construction presented in the manuscript; we do not assert the existence of such potentials for arbitrary learning dynamics or models. The derivation proceeds by invoking singular learning theory once this analyticity condition is met. To strengthen the presentation and address the referee's concern regarding explicit conditions, we will add a new subsection in the revised manuscript that delineates the precise mathematical conditions under which the effective potential is analytic for non-equilibrium dynamics. Furthermore, we will include a concrete example using a simple singular model (such as a reduced-rank regression) under finite-time gradient descent to illustrate a case where the effective potential is analytic, thereby substantiating the applicability of the framework. revision: partial

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity detected

full rationale

The abstract and described construction formulate a finite-time WBIC analogue by substituting an effective ensemble generated under a resource constraint, then conditionally invoke standard singular learning theory to obtain the resource-dependent RLCT when an analytic effective potential exists. This is an application of an external framework rather than a reduction of the central estimator to a fitted parameter, self-definition, or load-bearing self-citation chain. The claim remains conditional and does not assert existence of the potential for arbitrary cases, leaving the derivation self-contained against external benchmarks.

Assumptions & free parameters 0 free parameters · 1 assumptions · 1 invented entities

Ledger populated from explicit statements in the abstract only; full paper would be required for exhaustive enumeration.

assumptions (1)

domain assumption Singular learning theory applies once the effective ensemble admits an analytic effective potential.
Invoked directly in the abstract to obtain the resource-dependent threshold.

invented entities (1)

effective ensemble generated by learning dynamics under a resource constraint
purpose: Replaces the equilibrium posterior to enable finite-time model selection
Introduced in the abstract as the central modeling step.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Non-Equilibrium Model Selection via Finite-Time Thermodynamics." pith.science (2026). https://pith.science/paper/QHQSKAXB

@misc{pith2026260616399,
  author       = {Pith},
  title        = {Pith review of: Non-Equilibrium Model Selection via Finite-Time Thermodynamics},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QHQSKAXB}},
  note         = {Machine review of arXiv:2606.16399}
}

read the original abstract

Information criteria such as WAIC and WBIC extend model selection to singular learning machines, but they are usually derived for equilibrium posteriors. We formulate a finite-time analogue of WBIC by replacing the equilibrium posterior with an effective ensemble generated by learning dynamics under a resource constraint. When this ensemble admits an analytic effective potential, singular learning theory yields a resource-dependent real log canonical threshold. The resulting estimator gives a computable thermodynamic contribution to time-bounded MDL and identifies the finite-time singular complexity relevant to the structural information measured by epiplexity.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

12 extracted references · 1 canonical work pages

[1]

Akaike, IEEE Transactions on Automatic Control19, 716 (1974)

H. Akaike, IEEE Transactions on Automatic Control19, 716 (1974)

1974
[2]

Schwarz, The Annals of Statistics6, 461 (1978)

G. Schwarz, The Annals of Statistics6, 461 (1978)

1978
[3]

Watanabe,Algebraic Geometry and Statistical Learn- ing Theory(Cambridge University Press, Cambridge, 2009)

S. Watanabe,Algebraic Geometry and Statistical Learn- ing Theory(Cambridge University Press, Cambridge, 2009)

2009
[4]

Watanabe, Journal of Machine Learning Research11, 3571 (2010)

S. Watanabe, Journal of Machine Learning Research11, 3571 (2010)

2010
[5]

Watanabe, Journal of Machine Learning Research14, 867 (2013)

S. Watanabe, Journal of Machine Learning Research14, 867 (2013)

2013
[6]

Hironaka, Annals of Mathematics79, 109 (1964)

H. Hironaka, Annals of Mathematics79, 109 (1964)

1964
[7]

Hironaka, Annals of Mathematics79, 205 (1964)

H. Hironaka, Annals of Mathematics79, 205 (1964)

1964
[8]

arXiv preprint arXiv:2601.03220 , year=

M. Finzi, S. Qiu, Y. Jiang, P. Izmailov, J. Z. Kolter, and A. G. Wilson, arXiv preprint arXiv:2601.03220 (2026)

work page arXiv 2026

Show all 12 references

[9]

Welling and Y

M. Welling and Y. W. Teh, inProceedings of the 28th International Conference on Machine Learning (ICML- 11)(2011) pp. 681–688

2011
[10]

Kusuoka and D

S. Kusuoka and D. W. Stroock, Journal of the Faculty of Science, University of Tokyo, Section IA, Mathematics 32, 1 (1985)

1985
[11]

E. Lau, Z. Furman, G. Wang, D. Murfet, and S. Wei, in Proceedings of the 28th International Conference on Arti- ficial Intelligence and Statistics, Proceedings of Machine Learning Research, Vol. 258 (2025) pp. 244–252

2025
[12]

G. Wang, J. Hoogland, S. van Wingerden, Z. Furman, and D. Murfet, inProceedings of the International Con- ference on Learning Representations(2025)

2025

Pith tools

Reviewed June 27, 2026 · model on record in the stance chip above.

[1] [1]

Akaike, IEEE Transactions on Automatic Control19, 716 (1974)

H. Akaike, IEEE Transactions on Automatic Control19, 716 (1974)

1974

[2] [2]

Schwarz, The Annals of Statistics6, 461 (1978)

G. Schwarz, The Annals of Statistics6, 461 (1978)

1978

[3] [3]

Watanabe,Algebraic Geometry and Statistical Learn- ing Theory(Cambridge University Press, Cambridge, 2009)

S. Watanabe,Algebraic Geometry and Statistical Learn- ing Theory(Cambridge University Press, Cambridge, 2009)

2009

[4] [4]

Watanabe, Journal of Machine Learning Research11, 3571 (2010)

S. Watanabe, Journal of Machine Learning Research11, 3571 (2010)

2010

[5] [5]

Watanabe, Journal of Machine Learning Research14, 867 (2013)

S. Watanabe, Journal of Machine Learning Research14, 867 (2013)

2013

[6] [6]

Hironaka, Annals of Mathematics79, 109 (1964)

H. Hironaka, Annals of Mathematics79, 109 (1964)

1964

[7] [7]

Hironaka, Annals of Mathematics79, 205 (1964)

H. Hironaka, Annals of Mathematics79, 205 (1964)

1964

[8] [8]

arXiv preprint arXiv:2601.03220 , year=

M. Finzi, S. Qiu, Y. Jiang, P. Izmailov, J. Z. Kolter, and A. G. Wilson, arXiv preprint arXiv:2601.03220 (2026)

work page arXiv 2026