{"id":"3b7869af-bc36-4729-a782-04307fd4ea9d","arxiv_id":"2506.16715","paper_version":3,"verdict":"REJECT","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":3,"one_line_summary":"Classifier accuracy versus input noise shows a sharp transition, and the transition width decreases as a power law (exponent about 0.27) with the number of model parameters for EfficientNet-type models.","lead":"This paper adds increasing amounts of Gaussian noise to images and measures how the accuracy of EfficientNet classifiers drops to zero at a 'cognition transition' noise level. The authors report that the width of this transition shrinks as a power law with model size, and argue this shows statistical-mechanics scaling applies to deep learning.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The headline power law is not yet established: the DDPM step t used as the noise variable is internally inconsistent with Eq. 5, and since sigma_t saturates near 1 at the claimed transition, the width scaling in t may not survive conversion to actual noise level.","rationale":"The paper reports a potentially interesting empirical regularity, and the Discussion honestly limits the measured scaling to EfficientNet and notes computational limits. However, the lead paragraph and abstract make a much broader claim: 'the principles of statistical mechanics hold for deep learning' and that one can extrapolate model sizes across a universality class. For that claim to hold, the observed width scaling must be a genuine finite-size scaling law in the noise control parameter. The most load-bearing and least secure assumption is that the DDPM step index t is an adequate scaling variable. This assumption is internally contradicted by the paper's own noise conversion: log(sigma_c) ~ 11 is incompatible with Eq. 5, and sigma_t saturates near 1 well before the claimed transition region, so the abscissa is not a three-decade noise scale. A power law in t could be an artifact of the saturating transformation. The fix is concrete: redo the width analysis in sigma_t (or an SNR variable) with multiple seeds and report a data collapse; if the power law survives, it becomes a credible empirical finite-size scaling result. As written, the overclaim warrants rejection, so the reader's REJECT verdict stands unchanged.","tokens_in":13168,"tokens_out":6948,"duration_ms":74565,"concrete_test":"Recompute the Fig. 4 width analysis using the actual Gaussian noise standard deviation as the control variable: for each model and each accuracy level pair (a_low, a_up), convert the DDPM step t to sigma_t = sqrt(1 - prod_{s=0}^t (1 - beta_s)) via Eqs. (2)-(4), define the width Delta_sigma = |sigma(a_up) - sigma(a_low)|, and re-fit log(Delta_sigma) versus log(#parameters), ideally with 3 or more seeds per model. If the slope changes materially, the error bars overlap zero, or the width range no longer spans decades, the reported 0.27 power law is an artifact of the step-index proxy rather than a finite-size scaling law in noise.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim, a finite-size scaling power law for the cognition transition, depends on the control variable used to measure transition width. The paper uses the DDPM forward-step index t, stating t_c ~ 1845.934 and 'log(sigma_c) ~ 11' (Section IV, Fig. 2 inset). But from Eqs. (2)-(5), beta_t = 1e-4 + 4e-6 t and sigma_t = sqrt(1 - alpha_bar_t), with alpha_bar_t = prod_{s=0}^t (1 - beta_s). For t_c = 1845.934, the summed variance is about 7.0, so alpha_bar_t ~ 0.0009 and sigma_t ~ 0.9995; hence log(sigma_c) is near 0, not 11. Moreover sigma_t is a saturating function of t: for t from roughly 10^3 to 5000, sigma_t stays within about 1e-3 of 1. Thus the abscissa range log(step) in (10,12) is not 'three decades of noise' in any Gaussian-noise amplitude; it is a tiny, nearly constant interval of sigma. A power law in t therefore does not imply a power law in the actual noise standard deviation, and the claimed unique critical noise may be an artifact of the t-to-sigma saturation. The finite-size scaling form F(L) ~ L^{-lambda} f((T-T_c)L^{1/nu}) is assumed but never tested by a data collapse, and the width exponent 0.27 is fit without reported error bars or seed variability. Because the quantitative core rests on this unvalidated proxy, the central claim is not supported as stated.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies the accuracy of image classifiers (EfficientNet family, both ImageNet-pretrained B0–B7 and custom small models trained on CIFAR100) as a function of Gaussian noise added via the DDPM forward process. The authors identify a sharp “cognition transition” from high accuracy to near-zero accuracy when the noise step t reaches a critical value, and they report that the width of this transition narrows with increasing model size according to a power law with slope ≈ 0.27 over approximately three decades of model size. They interpret this as finite-size scaling, claim a unique critical noise for all models, and suggest that the result can be used to extrapolate robustness to larger models without full training. The paper is written as an empirical physics-style scaling study.","tokens_in":13516,"tokens_out":4372,"duration_ms":45093,"significance":"If the reported power law were correct, it would provide a quantitative finite-size scaling relation for classifier robustness under additive Gaussian noise, with practical value for model sizing and an interesting bridge between statistical mechanics and deep learning. The paper also attempts to use a level-set method for measuring transition widths, which, if validated, could be a useful tool. However, the central quantitative claim rests on the choice of the DDPM step index t as the noise variable, and the manuscript contains an internal inconsistency in the conversion between t and the actual noise standard deviation sigma. Because the scaling may be an artifact of the t-to-sigma saturation, the significance of the result, while potentially high, is not currently established. Strengths include the use of a controlled, well-defined noise process (DDPM) and a clear statement of the finite-size scaling hypothesis, but the analysis lacks error bars, seed variability, and a data-collapse test.","major_comments":[{"comment":"The claim that the unique critical step t_c = 1845.934 corresponds to log(sigma_c) ≈ 11 is internally inconsistent with the stated noise schedule. Using beta_t = 1e-4 + 4e-6 t, the cumulative variance at t_c is sum_{s=0}^{t_c} beta_s ≈ 7.0, giving alpha_bar(t_c) ≈ e^{-7} and sigma(t_c) = sqrt(1 - alpha_bar) ≈ 0.9991, so log(sigma_c) is near 0, not 11. More importantly, sigma(t) is a saturating function of t: between t = 1000 and t = 2000 it changes from about 0.94 to 0.9998. Thus the transition region in t spans only a tiny interval of actual noise amplitude sigma, and a power law in the step index t does not imply a power law in the physical noise level. The claimed “three decades of noise” (lead paragraph) is actually three decades of model size, and the quantitative core of the paper—the width scaling—is measured in the proxy variable t without demonstrating that it survives conversion to sigma.","section":"Section IV, Fig. 2 and Eqs. (2)–(5)"},{"comment":"The finite-size scaling form F(L) ∝ L^{-lambda} f((T - T_c) L^{1/nu}) is asserted but never tested by a data collapse. The exponent 0.27 is obtained from a log-log fit of transition width versus parameter count, but no error bars, number of seeds, or uncertainty analysis are reported, and the fit range includes only about ten model sizes. A genuine scaling hypothesis should collapse all the accuracy-versus-noise curves onto a single master curve when plotted against (t - t_c) L^{1/nu}; this is not shown. Without such a collapse, the reported power law could reflect the specific choice of level sets or the accidental alignment of a small number of points, rather than a universal scaling relation.","section":"Section II and Fig. 4"},{"comment":"The assertion that there is a single critical noise value t_c ≈ 1845.934 for all EfficientNet models is stated without any description of how this value was determined. In standard finite-size scaling, the effective transition point typically shifts with system size, approaching the thermodynamic limit as L grows. If the transition center is claimed to be exactly size-independent, that is a special and non-generic property that needs justification; otherwise the nested-interval level-set measurement of the width may be dominated by the arbitrarily chosen accuracy levels rather than by a well-defined scaling variable. The uniqueness of t_c is load-bearing for the interpretation of the width scaling, and it is asserted rather than demonstrated.","section":"Section IV (Fig. 2 inset)"}],"minor_comments":[{"comment":"The lead paragraph states that the authors “can determine scaling behavior over three decades of noise,” but the scaling in Fig. 4 is over model size (parameter count). Please clarify which quantity spans the three decades; if both are intended, the text should distinguish them explicitly.","section":"Lead paragraph and Section IV, Fig. 3"},{"comment":"In the accuracy definition, the text defines y as the predicted label and ŷ as the true label, but the indicator function is written as 1(ŷ_i = y_i), which is symmetric. Please correct the notation so that the roles of predicted and true labels are unambiguous.","section":"Section III.B.1, Eq. (6)"},{"comment":"The description of the custom small models as “EfficientNet-compatible sizes” is vague. Please specify the exact architecture (depth, width, resolution) and training hyperparameters for these models, since they form the majority of the data points in the scaling fit.","section":"Section III.B"},{"comment":"The FLOPS column appears to use units of GFLOPs (or similar), but the units are not stated. Please add explicit units and a brief definition of how FLOPS is computed.","section":"Table I"},{"comment":"Several references in the bibliography (e.g., references to MNIST and Fashion-MNIST in the reference list) are not cited in the text. Please ensure that all listed references are actually used and cited where relevant.","section":"General"}],"recommendation":"reject","confidential_remarks":"The internal inconsistency between the reported log(sigma_c) ≈ 11 and the value computed from the paper's own Eqs. (2)–(5) indicates that the analysis may be using the DDPM step index as a proxy for noise without checking its mapping to the actual noise standard deviation. Given that sigma(t) saturates near unity in the transition region, the reported power-law width scaling could be an artifact of the proxy. This is a load-bearing issue that cannot be fixed by minor edits; it would require redefining the control variable in terms of sigma (or a signal-to-noise ratio) and repeating the entire scaling analysis. I therefore recommend rejection, though I would encourage the authors to revisit the analysis with the appropriate noise metric, as the underlying question is of interest."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Martin,\n\nOne thing to know: there's a plausible empirical finding here — the transition width in classification accuracy under added Gaussian noise shrinks as a power law with model size for EfficientNet, slope around 0.27 — but the paper's own equations contradict the reported noise level, and without seed variability, error bars, or code, the scaling law is not yet verified.\n\nWhat is new: the specific measurement of that width exponent for a family of convolutional classifiers over a couple of decades of parameter count. The level-set approach to define transition width is a decent idea, and the authors admit at the end that they have not established universality beyond one architecture family. That honesty counts for something.\n\nThe soft spots are the problem. The exponent is fitted to the same data used to claim the scaling law, with no held-out validation or data collapse to test the finite-size scaling ansatz. The paper reports t_c = 1845.934 and 'log(sigma_c) ≈ 11' for the same transition. Using their own Eq. (5), the cumulative noise variance at that step is about 7, so sigma is about 0.9995 and log10(sigma) is near 0, not 11. And sigma saturates to 1 quickly, so the entire transition region maps to a sliver of actual noise strength. That means the power law in t may not survive conversion to the physical noise variable the paper claims to be studying. The claim that 'the principles of statistical mechanics hold for deep learning' is far beyond anything this evidence supports.\n\nWho gets value: a robustness-scaling researcher might look at the raw curve as a hint, but they'd have to redo the experiments properly. As is, I wouldn't cite it, and I wouldn't send it to reviewers. If the authors fix the noise conversion, add multiple seeds and architectures, and present the scaling as an empirical fit, it could become a decent short paper. Until then, it's a sloppy draft with an interesting core.\n\nBest,","headline":"A potentially useful empirical scaling law, undermined by a bad noise conversion and a lack of reproducibility.","tokens_in":14034,"tokens_out":6561,"would_cite":false,"duration_ms":67683,"reading_group":"no","serious_thinker":"no","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The authors show that the accuracy-versus-noise transition narrows as a power law in model size, with exponent 0.27 over three decades.","keywords":["cognition transition","finite-size scaling","scaling laws","critical exponents","universality class","EfficientNet","image classification","Gaussian noise"],"falsifier":"Apply the same noise schedule and level-set width measurement to a second architecture family; if the log-log plot of transition width versus parameter count is not a straight line with a stable slope, the claimed universal scaling fails. As an arithmetic check, compute $\\sigma = \\sqrt{1-\\bar\\alpha_t}$ from the stated schedule at $t_c = 1845.934$ and compare $\\log \\sigma$ with the quoted value 11.","tokens_in":12893,"feed_emoji":"📉","tokens_out":11653,"duration_ms":103110,"temperature":0.7,"pith_summary":"The paper tries to establish that the loss of image-classification accuracy under added Gaussian noise is a sharp \"cognition transition\" and that, as models grow, this transition narrows according to a finite-size scaling law. Using the EfficientNet family (roughly $10^4$ to $10^7$ parameters), the authors corrupt test images with a diffusion-style forward noise process and record accuracy as a function of noise step. They find a power-law narrowing of the transition width with a slope of about 0.27 across three decades of model size, with a common critical noise for all sizes. If correct, this gives a practical recipe: measure small models cheaply and extrapolate to larger ones, and it suggests that statistical-mechanics universality classes could organize model robustness.","feed_headline":"Accuracy cliff sharpens with model size by a 0.27 power law","feed_subtitle":"The cliff's width falls as parameters grow to the 0.27 power, so small, cheap models can predict larger ones.","key_machinery":"The central object is the finite-size scaling ansatz $F(L) \\propto L^{-\\lambda} f((T-T_c)L^{1/\\nu})$, applied with $L$ the number of trainable parameters and $T$ the diffusion noise step. The transition width is extracted with the level-set method: nested accuracy thresholds such as $(0.1,0.9)$ and $(0.2,0.8)$ define noise-step intervals, and the interval width $\\Delta = |\\beta(a_{\\text{up}})-\\beta(a_{\\text{low}})|$ is plotted against model size. Noise is injected through the forward process of denoising diffusion probabilistic models with the linear schedule $\\beta_t = 10^{-4} + 4\\times 10^{-6} t$ over 5000 steps, giving Gaussian corruption with analytic marginals. This machinery produces the central plot: a double-logarithmic straight line of slope 0.27 for transition width versus parameter count.","core_discovery":"The central discovery is that the transition from good classification to chance-level performance under controlled Gaussian noise behaves like a finite-size critical phenomenon. For a fixed family of models, larger networks perform better and the accuracy transition becomes sharper, while the critical noise level remains fixed (the authors report $t_c = 1845.934$ in diffusion steps for all EfficientNet sizes). The sharpness, defined as the width $\\Delta$ of nested accuracy level sets, is claimed to obey a power law in the number of parameters, $\\Delta \\sim N^{-0.27}$, across three decades. The authors interpret this as evidence for the finite-size scaling form $F(L) \\sim L^{-\\lambda} f((T-T_c)L^{1/\\nu})$ and take it to indicate universal behavior within a class of similarly built models.","pith_inferences":["If the scaling extends beyond EfficientNet, applying the same protocol to other architecture families would be a direct test; a shared exponent would mean robustness to Gaussian noise is governed mainly by parameter count, not by architectural detail.","The noise axis itself deserves scrutiny: the quoted $\\log \\sigma_c \\approx 11$ does not match the analytic $\\sigma = \\sqrt{1-\\bar\\alpha_t}$ from Eq. 5 at the reported $t_c = 1845.934$, so the quantitative position of the transition may need a corrected conversion.","Beyond classification accuracy, the same finite-size scaling could be probed on regression tasks or calibration error to ask whether the cognition transition and its exponent are universal across tasks."],"forward_implications":["If the power law is correct, a practitioner can measure the transition on a small, cheaply trained model and extrapolate to the model size needed for a target robustness, avoiding full large-scale training.","The finding that the critical noise is the same across EfficientNet sizes implies that scaling up model size sharpens the transition rather than moving where failure begins.","The reported exponent 0.27 gives a quantitative baseline: model families in the same universality class should reproduce this slope, and deviations would signal a different class.","The level-set width protocol offers a systematic way to compare robustness across models without depending on the exact accuracy at any one noise level."],"supporting_citations":[{"why":"Defines the forward noising process whose Gaussian perturbation steps are used as the noise-intensity axis.","marker":"[4]"},{"why":"Provides the linear noise schedule and the analytic marginal sampling used to corrupt images and compute the noise level.","marker":"[24]"},{"why":"Defines the EfficientNet architecture and compound scaling that fix the system-size axis in terms of parameter count.","marker":"[40]"},{"why":"Supplies the finite-size scaling concept that predicts transition sharpening with increasing system size.","marker":"[7]"},{"why":"Provides the statistical-mechanics framework of order parameters, critical exponents, and universality that motivates the analysis.","marker":"[5]"},{"why":"Motivates the level-set construction used to define nested accuracy thresholds and measure transition width.","marker":"[48]"},{"why":"Companion level-set reference for extracting the width of the transition and its scaling behavior.","marker":"[49]"},{"why":"Earlier study by the same authors establishing the cognition transition and its dependence on model size, which this paper extends.","marker":"[6]"},{"why":"ImageNet dataset used for evaluating the pretrained EfficientNet models.","marker":"[38]"},{"why":"CIFAR dataset used to train the small EfficientNet models from scratch.","marker":"[39]"}],"fun_headline_variants":["Noise cliff sharpens with model size: 0.27 power law","Accuracy drop under noise sharpens like N^-0.27","Cognition transition: sharper cliffs for larger models","Model size sets accuracy cliff sharpness via 0.27 scaling","Noise-induced accuracy cliff follows N^-0.27 scaling"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The argument assumes that accuracy-versus-noise curves for models of different sizes all line up according to one statistical scaling law, with a particular step-counting scheme for Gaussian noise as the measure of noise; this has been tested only on one model family.","fun_headline_variants_meta":{"raw":{"variants":["Noise cliff sharpens with model size: 0.27 power law","Accuracy drop under noise sharpens like N^-0.27","Cognition transition: sharper cliffs for larger models","Model size sets accuracy cliff sharpness via 0.27 scaling","Noise-induced accuracy cliff follows N^-0.27 scaling"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000379,"raw_usage":{"total_tokens":2015,"prompt_tokens":947,"completion_tokens":1068,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":563,"completion_tokens_details":{"reasoning_tokens":981}},"tokens_in":563,"tokens_out":1068,"duration_ms":10420,"temperature":1.0,"reasoning_tokens":981,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T19:20:01.740571+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Apply the same noise schedule and level-set width measurement to a second architecture family; if the log-log plot of transition width versus parameter count is not a straight line with a stable slope, the claimed universal scaling fails. As an arithmetic check, compute $\\sigma = \\sqrt{1-\\bar\\alpha_t}$ from the stated schedule at $t_c = 1845.934$ and compare $\\log \\sigma$ with the quoted value 11.","supporting_citations":[{"cited_title":"Sohl-Dickstein , author E","cited_arxiv_id":null,"evidence_quote":"Defines the forward noising process whose Gaussian perturbation steps are used as the noise-intensity axis."},{"cited_title":"Ho , author A","cited_arxiv_id":null,"evidence_quote":"Provides the linear noise schedule and the analytic marginal sampling used to corrupt images and compute the noise level."},{"cited_title":"Binder ,\\ title title Finite size effects at phase transitions , \\ in\\ @noop booktitle Computational Methods in Field Theory ,\\ editor edited by\\ editor H","cited_arxiv_id":null,"evidence_quote":"Supplies the finite-size scaling concept that predicts transition sharpening with increasing system size."},{"cited_title":"Huang ,\\ https://books.google.de/books?id=8bkBJgAACAAJ title Introduction to Statistical Physics, Second Edition \\ ( publisher Taylor & Francis ,\\ year 2009 ) NoStop","cited_arxiv_id":null,"evidence_quote":"Provides the statistical-mechanics framework of order parameters, critical exponents, and universality that motivates the analysis."},{"cited_title":"Seidler \\ and\\ author M","cited_arxiv_id":null,"evidence_quote":"Earlier study by the same authors establishing the cognition transition and its dependence on model size, which this paper extends."},{"cited_title":"Deng , author W","cited_arxiv_id":null,"evidence_quote":"ImageNet dataset used for evaluating the pretrained EfficientNet models."},{"cited_title":"Krizhevsky , author G","cited_arxiv_id":null,"evidence_quote":"CIFAR dataset used to train the small EfficientNet models from scratch."}],"review_version":2}