Pith. sign in

REVIEW 2 major objections 1 minor 21 references

Reviewed by Pith at T0; open to challenge.

T0 means a machine referee read the full paper against a public rubric. The mark states how deep the mechanical check went, never who wrote it. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.3

Small scaling exponents in LLMs indicate unsustainable energy use even after pedestal correction

2026-06-26 00:02 UTC pith:HDQBHCQQ

load-bearing objection The paper asserts that small LLM scaling exponents signal energy unsustainability even after pedestal correction, but supplies no energy-budget math or projections to support the inference. the 2 major comments →

arxiv 2606.24504 v1 pith:HDQBHCQQ submitted 2026-06-23 cs.AI

On the Smallness of the Large Language Models Scaling Exponents

Sauro Succi , Peter V. Coveney , Alex Hansen This is my paper
classification cs.AI
keywords LLM scaling exponentsenergy sustainabilitypedestal effectloss functionturbulence analogylarge language modelsscaling lawsdata smoothness
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper argues that the small scaling exponents observed for large language models point to an unsustainable regime in energy consumption as models grow. It tests whether this smallness stems from a numerical bias called the pedestal effect, in which the loss function fails to reach zero even with infinite data, and concludes that the correction leaves the unsustainability intact. The authors further note that the smoothness or roughness of training data influences the exponents, drawing on an analogy to phenomenological models of fluid turbulence. A sympathetic reader would care because the claim directly ties observed scaling behavior to long-term feasibility of current LLM approaches.

Core claim

The central claim is that the smallness of the scaling exponents in current LLM applications signals an unsustainable energy regime. Attributing the smallness to the pedestal effect arising from a non-zero loss function at infinite data does not remove the unsustainability issue. Effects of data smoothness or roughness on the exponents are discussed via an analogy with fluid turbulence models.

What carries the argument

The pedestal effect (non-zero asymptotic loss) together with the turbulence-phenomenology analogy for how data smoothness affects scaling exponents.

Load-bearing premise

That the observed small scaling exponents directly imply an unsustainable energy regime for LLMs, with the pedestal effect and turbulence analogy providing sufficient grounds to maintain this conclusion without additional quantitative energy modeling.

What would settle it

A calculation showing that, once the pedestal effect is included, the effective scaling exponents become large enough for energy cost to grow sublinearly with performance improvements.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • The small exponents keep LLM training and inference in an unsustainable energy regime.
  • Correcting for a non-zero loss at infinite data leaves the unsustainability conclusion unchanged.
  • Data smoothness or roughness modulates the scaling exponents in a manner analogous to turbulence models.
  • Current LLM applications therefore face fundamental limits tied to energy resources.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Quantitative energy-consumption modeling beyond the scaling-exponent argument would be needed to size the practical impact.
  • The turbulence analogy suggests that changes in data preprocessing or architecture could alter exponents without changing model size.
  • Similar smoothness-based arguments might apply to scaling in other complex systems such as physical simulations or biological networks.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 1 minor

Summary. The manuscript argues that observed scaling exponents in LLMs indicate an unsustainable energy regime, that the pedestal effect (non-zero asymptotic loss) does not remove this unsustainability, and that an analogy to phenomenological turbulence models can account for the influence of data smoothness/roughness on the exponents.

Significance. The question of whether LLM scaling is energetically sustainable is relevant to the field. If the central inference were supported by explicit energy-budget calculations or falsifiable projections, the work could contribute to that discussion; as written, the absence of such grounding limits its potential impact.

major comments (2)
  1. [Abstract] Abstract and main text: the claim that small scaling exponents directly imply an unsustainable energy regime is asserted without any energy-budget calculation, FLOPs projection, power-draw estimate, or comparison against global resources; this inference is load-bearing for the unsustainability conclusion.
  2. [Abstract] Abstract and main text: the statement that the pedestal effect 'does not remove the unsustainability issue' is made without showing how the corrected exponents still produce an unsustainable regime; the turbulence analogy addresses data smoothness but supplies no quantitative energy link.
minor comments (1)
  1. Notation for the scaling exponents and the pedestal term should be defined explicitly at first use to aid readability.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the review. The manuscript analyzes implications of reported LLM scaling exponents for energy use, without performing new resource projections. We address the major comments point by point below.

read point-by-point responses

  1. Referee: [Abstract] Abstract and main text: the claim that small scaling exponents directly imply an unsustainable energy regime is asserted without any energy-budget calculation, FLOPs projection, power-draw estimate, or comparison against global resources; this inference is load-bearing for the unsustainability conclusion.

    Authors: The manuscript discusses how the smallness of scaling exponents reported in the LLM literature mathematically implies an unsustainable regime, as marginal loss reductions would require disproportionately large increases in scale. No explicit energy-budget calculations, FLOPs projections, or global-resource comparisons are provided because the work is an analysis of scaling-law consequences rather than a computational forecasting study. The inference follows directly from the functional form of the scaling relations themselves. revision: no

  2. Referee: [Abstract] Abstract and main text: the statement that the pedestal effect 'does not remove the unsustainability issue' is made without showing how the corrected exponents still produce an unsustainable regime; the turbulence analogy addresses data smoothness but supplies no quantitative energy link.

    Authors: The text shows that even after accounting for a non-zero asymptotic loss (pedestal), the effective exponents remain small and continue to signal the same regime. The turbulence analogy is introduced solely to interpret how data roughness modulates the observed exponents, following established phenomenological parallels; it is not intended to supply an energy calculation. The unsustainability link continues to rest on the magnitude of the (corrected) exponents. revision: no

Circularity Check

0 steps flagged

No circularity; unsustainability claim is an interpretive assertion, not a self-referential derivation

full rationale

The paper asserts that observed small scaling exponents indicate an unsustainable energy regime and that the pedestal effect does not remove this issue, with a turbulence analogy for data smoothness. No equations, fitted parameters renamed as predictions, or self-citations are present in the abstract or described chain. The central claim does not reduce by construction to its inputs; it is an external inference from scaling data to energy implications without quantitative modeling shown. This is a standard non-finding for papers whose core move is commentary rather than a closed derivation loop.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Based solely on the abstract; the central argument assumes the validity of reported small scaling exponents and the pedestal concept without specifying fitted values or new axioms in the provided text.

pith-pipeline@v0.9.1-grok · 5612 in / 946 out tokens · 36014 ms · 2026-06-26T00:02:27.185433+00:00 · methodology

0 comments
read the original abstract

We discuss reasons why the scaling exponents of current Large Language Models (LLMs) applications are indicating an unsustainable regime in terms of energy resources. We further show that attributing the smallness of such exponents to a numerical bias due to the neglect of a non-zero value of the loss function in the limit of infinite data (``pedestal effect") does not remove the unsustainability issue. Finally, the effects of the smoothness (roughness) of the data on the scaling exponents is commented upon based on an analogy with phenomenological models of fluid turbulence.

Figures

Figures reproduced from arXiv: 2606.24504 by Alex Hansen, Peter V. Coveney, Sauro Succi.

Figure 1
Figure 1. Figure 1: The running scaling exponent as a function of 1 [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: The critical boundary Ncrit(p) = ( 1−p p ) 3 between C and A scaling regions, as a function of the pedestal p. The C-scaling is only relevant below the critical boundary, hence for relatively small datasets size, unless the pedestal is made unrealistically small. First, note that the SK relation refers to concrete LLMs practice, hence it is fully consistent with the pseudo-metric perspective discussed abov… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

21 extracted references · 5 canonical work pages · 3 internal anchors

  1. [1]

    Attention Is All You Need

    Vaswani, Ashish; Shazeer, Noam; Parmar, Niki; Uszkoreit, Jakob; Jones, Llion; Gomez, Aidan N; Kaiser, Lukasz; Polosukhin, Illia (December 2017). ”Attention is All you Need”. In I. Guyon and U. Von Luxburg and S. Ben- gio and H. Wallach and R. Fergus and S. Vishwanathan and R. Garnett (ed.), 31st Conference on Neural Information Processing Systems (NIPS). ...

    work page internal anchor Pith review Pith/arXiv arXiv 2017

  2. [2]

    J Jumper, R Evans, A Pritzel, T Green, M Figurnov, O Ronneberger, K Tunyasuvunakool, R Bates, A ˇZ´ ıdek, A Potapenko, A Bridgland, C Meyer, S A. A. Kohl, AJ. Ballard, A Cowie, B Romera-Paredes, S Nikolov, R Jain, J Adler, T Back, S Petersen, D Reiman, E Clancy, M Zielinski, . . . Demis Hassabis, Highly accurate protein structure prediction with AlphaFold...

    2021

  3. [3]

    Lecun, J

    Y. Lecun, J. Bengio and G. Hilton, Deep Learning, Nature 521 (7553), 436-444 (2015)

    2015

  4. [4]

    Scaling Laws for Neural Language Models

    J. Kaplan et al, Scaling laws for neural language models, arXiv:2001.08361v1 [cs.LG] 23 Jan 2020

    work page internal anchor Pith review Pith/arXiv arXiv 2001

  5. [5]

    Y Bahri, E Dyer, J Kaplan, J Lee, U Sharma, Explaining neural scal- ing laws, Proceedings of the National Academy of Sciences 121 (27), e2311878121, (2024)

    2024

  6. [6]

    Z. Ji, M. Jiang, A systematic review of electricity demand for large language models: evaluations, challenges, and solutions, Renewable and Sustainable Energy Reviews, Volume 225, 116159 (2026)

    2026

  7. [7]

    The wall confronting large language models

    P.V. Coveney and S. Succi, The wall confronting large language models, arXiv preprint arXiv:2507.19703 (2025)

    work page Pith review arXiv 2025

  8. [8]

    PV Coveney, ER Dougherty, RR Highfield, Big data need big theory too, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences; 374:20160153 (2016)

    2016

  9. [9]

    S.Succi, PV Coveney, Big data: the end of the scientific method? Philo- sophical Transactions of the Royal Society A; 377:201801 (2019)

    2019

  10. [10]

    Hansen, S

    A. Hansen, S. Succi, A new kind of science, Frontiers in physics, 13, 1760758 (2025)

    2025

  11. [11]

    K Vafa, PG Chang, A Rambachan, S Mullainathan, What has a founda- tion model found? using inductive bias to probe for world models, arXiv preprint arXiv:2507.06952 (2025)

    work page arXiv 2025

  12. [12]

    Y Liu, Z Liu, J Gore, Superposition yields robust neural scaling, Advances in Neural Information Processing Systems, 38, 159269-159305, 20 (2026) 10

    2026

  13. [13]

    Training Compute-Optimal Large Language Models

    J. Hoffmann et al, Training Compute-Optimal Large language Models, arXiv:2203.15556v1 [cs.CL] 29 Mar 2022

    work page internal anchor Pith review Pith/arXiv arXiv 2022

  14. [14]

    Sharma and J

    U. Sharma and J. Kaplan, Scaling Laws from the Data Manifold Dimension, Journal of Machine Learning Research 23, 1-34 (2024)

    2024

  15. [15]

    A Ansuini, A Laio, JH Macke, D Zoccolan, Intrinsic dimension of data representations in deep neural networks, Advances in Neural Information Processing Systems 32 (2019)

    2019

  16. [16]

    Bellman, Dynamic Programming, Princeton University Press, Prince- ton, NJ, (1957)

    R. Bellman, Dynamic Programming, Princeton University Press, Prince- ton, NJ, (1957)

    1957

  17. [17]

    Frisch, Turbulence, the legacy of A.N

    U. Frisch, Turbulence, the legacy of A.N. Kolmogorov, Cambridge U.P., 1995

    1995

  18. [18]

    Briscolini, P

    M. Briscolini, P. Santangelo, S. Succi and R. Benzi, Extended self-similarity in the numerical simulation of three-dimensional homogeneous flows, Phys. Rev. E, 50, 3, R1745 (1994)

    1994

  19. [19]

    Succi, Sailing the ocean of complexity: lessons from the physics-biology interface, Oxford: Oxford University Press (2022)

    S. Succi, Sailing the ocean of complexity: lessons from the physics-biology interface, Oxford: Oxford University Press (2022)

    2022

  20. [20]

    A. N. Kolmogorov, The local structure of turbulence in incompressible viscous fluid for very large Reynolds numbers, Dokl. Akad. Nauk SSSR, 30, 9-13, reprinted in Proc. Roy/ Soc. London 434, 9-13 (1991)

    1991

  21. [21]

    Frisch, P

    U. Frisch, P. Sulem, M. Nelkin, J. Fluid Mech., vol. 87, Aug. 29, 719-736 (1978). 11

    1978