Thermodynamic lower bounds are approximated for exact and SGD linear regression, producing energy-aware scaling laws for optimal training dataset size given a target generalization error.
The Entropy of Floating-Point Numbers
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
abstract
Here we present an analytic approximation for the entropy of floating-point numbers, along with bounds on the error of this approximation. It is well-known that the differential entropy is tightly linked to the discrete entropy of a uniformly quantized random variable. Our approximation uncovers a different quantity that provides this link for floating-point quantization. Additionally, we prove that the entropy of a floating-point quantized random variable is approximately unchanged under scaling. Closed-form expressions for the floating-point entropy of common distributions are provided and compared to exact results.
fields
cond-mat.stat-mech 1years
2026 1verdicts
UNVERDICTED 1representative citing papers
citing papers explorer
-
The Thermodynamic Costs of Simple Linear Regression
Thermodynamic lower bounds are approximated for exact and SGD linear regression, producing energy-aware scaling laws for optimal training dataset size given a target generalization error.