Pith. sign in

REVIEW 3 major objections 5 minor 1 cited by

Squared-loss surrogates hit a conditional-mean barrier that erases the variability science needs.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-12 15:49 UTC pith:PLY5OKRB

load-bearing objection Classical MSE→conditional-mean fact packaged as a usable SciML diagnostic and loss prescription; moderate novelty, clear argument, thin but honest empirics. the 3 major comments →

arxiv 2605.28076 v3 pith:PLY5OKRB submitted 2026-05-27 stat.ML cs.NAmath.NAnlin.CDphysics.data-an

Diagnosing the conditional-mean barrier in scientific machine-learning surrogates

Junfeng Chen This is my paper
classification stat.ML cs.NAmath.NAnlin.CDphysics.data-an
keywords conditional-mean barrierscientific machine learningsurrogate modelssquared lossstochastic closuresLorenz-96residual diagnosticsconditional law
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

In many scientific prediction tasks, coarse graining or partial observation turns a one-to-one map into a one-to-many conditional law: the same input can produce a cloud of plausible outputs. Deterministic machine-learning surrogates trained by ordinary squared loss still converge to a mathematically well-defined object—the conditional mean—but that single number discards the scatter that often carries the physically relevant fluctuations. The paper names this the conditional-mean barrier and supplies a practical diagnostic: residual-feature orthogonality checks whether the leftover error is uncorrelated with the features that matter, while effect-size measures decide whether that leftover scatter is large enough to matter for the task. The same analysis shows why simply making the network stochastic does not fix the problem under paired squared loss—the objective itself penalizes model variance and drives every realization back to the mean. When the diagnostic flags the barrier, the prescription is clear: switch to a loss that scores richer features of the conditional distribution. Controlled two-branch examples and a two-scale Lorenz-96 closure demonstrate that the diagnostic correctly identifies the barrier, that deterministic closures suppress collective fluctuation statistics in long rollouts, and that a minimal likelihood-based stochastic model can restore much of the missing variability.

Core claim

Deterministic surrogates trained by paired squared loss learn the conditional mean and therefore miss task-relevant variability in the underlying conditional law—the conditional-mean barrier. Residual-feature orthogonality together with effect-size diagnostics can separate this irreducible conditional scatter from ordinary deterministic underfitting, and the same squared-loss objective forces even stochastic predictors back to the mean.

What carries the argument

The residual-feature orthogonality and effect-size diagnostic: after a surrogate is fit, residuals are tested for orthogonality to chosen residual features; large residual effect size with near-orthogonality signals irreducible conditional variability rather than under-fitting, diagnosing the barrier.

Load-bearing premise

That the residual features and effect-size thresholds chosen on the two numerical examples actually capture the variability that matters for the scientific task, rather than merely residual scatter relative to those particular features.

What would settle it

On a new one-to-many scientific surrogate, compute residual-feature orthogonality and effect size; if residuals remain large and nearly orthogonal yet a richer likelihood or proper scoring rule still fails to recover the missing fluctuation statistics that the paper’s Lorenz-96 experiment recovered, the diagnostic-plus-prescription claim fails.

Watch this falsifier — get emailed when new claim-graph text bears on it.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper formulates the conditional-mean barrier: after coarse-graining or partial observation, many SciML prediction tasks are one-to-many, so deterministic surrogates trained by paired squared loss recover the conditional mean E[Y|X] while missing task-relevant variability in the conditional law. It proposes a diagnostic that combines residual-feature orthogonality with effect-size measures to separate deterministic underfitting from irreducible conditional scatter, and notes that stochastic predictors under the same paired MSE objective are driven back to the conditional mean because model variance is penalized. The modeling prescription is that when residual variability matters, the loss must score richer features of the conditional law (e.g., likelihood-based stochastic-scale models). Reproducible studies on a controlled two-branch law and a two-scale Lorenz-96 closure illustrate barrier identification, suppression of collective fluctuation statistics under deterministic closures in rollout, and partial recovery of variability by a minimal stochastic-scale model.

Significance. If the diagnostic is used as intended, the paper gives SciML practitioners a concrete way to decide when a well-fitting deterministic surrogate is still the wrong object for the scientific task, and a clear loss-level prescription rather than an architecture-level one. The core L2 fact (population minimizer is the conditional mean; paired MSE penalizes predictor variance) is classical, but naming the barrier, packaging residual-feature orthogonality with effect-size, and demonstrating the rollout consequence on a two-scale Lorenz-96 closure are useful contributions for the community. Strengths include an explicit, non-circular definition of the barrier from the known population minimizer, a falsifiable residual diagnostic, and reproducible numerical studies. The main limitation is transfer: whether the chosen residual features and effect-size thresholds reliably flag task-relevant variability beyond the two examples.

major comments (3)
  1. [Diagnostic framework / residual-feature orthogonality] The diagnostic framework (residual-feature orthogonality plus effect-size) is load-bearing for the claim that one can operationally identify the barrier rather than only restate the L2 fact. The residual feature bank and effect-size thresholds are free parameters of the method. The manuscript needs a clearer statement of how those features are chosen for a new problem, what constitutes a decisive effect size, and how the procedure avoids mistaking structured misspecification for irreducible conditional variability. Without that guidance or a sensitivity study, the two numerical examples remain illustrations rather than a transferable operational test.
  2. [Two-scale Lorenz-96 closure / rollout statistics] The phrase task-relevant variability is central to the modeling prescription but is not given an independent operational definition beyond residual scatter relative to the chosen features. In the Lorenz-96 closure study, the paper should state explicitly which collective fluctuation statistics are the scientific target (e.g., variance, spectra, extremes of the resolved variables in rollout) and show that the residual features used in the diagnostic are aligned with those targets, not only with one-step residual magnitude. Otherwise the claim that the diagnostic identifies scientifically important missing variability rests on the same free feature choice.
  3. [Abstract / introduction; paired squared-loss argument] The consequence that stochastic outputs under paired squared loss do not overcome the barrier is correct but standard (the excess risk includes the model variance term). The manuscript should more carefully separate this pedagogical clarification from the novel contribution (the residual diagnostic and the SciML prescription). As written, the abstract and introduction can be read as presenting the stochastic-under-MSE fact as a primary result; tightening the novelty claim would strengthen the paper without changing the math.
minor comments (5)
  1. [Diagnostic framework] Define residual-feature orthogonality and the effect-size statistic with explicit formulas early (population and sample versions), and keep notation consistent between the two-branch and Lorenz-96 sections.
  2. [Numerical studies] State training protocol, architecture, and hyperparameter choices for the deterministic and stochastic surrogates in enough detail for independent reimplementation; point to the reproducibility package if that is the intended source of truth.
  3. [Controlled two-branch law] In the two-branch law example, report both residual diagnostics and a simple visualization of the recovered conditional law (or lack thereof) so readers can see the barrier without relying only on scalar effect sizes.
  4. [Stochastic-scale model] Clarify whether the likelihood-based stochastic-scale model is trained with a proper scoring rule for the full conditional (or a parametric scale family) and how that differs from injecting noise into an MSE-trained mean model.
  5. [Figures / rollout panels] Minor polish: ensure figure captions state what is being compared (truth vs deterministic closure vs stochastic-scale) and what time horizon the rollout statistics cover.

Circularity Check

0 steps flagged

No significant circularity: the barrier is the classical L2 population minimizer, and the diagnostics/experiments are independent checks rather than tautologies.

full rationale

The paper’s central claim—that paired squared loss has population minimizer equal to the conditional mean E[Y|X], so deterministic (and even stochastic) predictors trained under that loss cannot recover non-mean features of the conditional law—is a standard, self-contained mathematical fact, not a fit or a self-citation chain. Residual-feature orthogonality and effect-size diagnostics are proposed post-fit checks that separate well-specified mean residuals from irreducible conditional scatter; they do not redefine the target they claim to detect. The two-branch synthetic law and two-scale Lorenz-96 closure are illustrative numerical studies with known or independently generated ground truth, not predictions forced by construction from fitted constants. Stochastic collapse under squared loss is again a direct consequence of the objective (model variance is penalized), not a circular re-labeling. No load-bearing uniqueness theorem, ansatz, or self-citation is required for the derivation. Framing the classical fact as a “conditional-mean barrier” and packaging diagnostics for SciML is contribution framing, not circular reduction of outputs to inputs. Score 0 is therefore warranted.

Axiom & Free-Parameter Ledger

4 free parameters · 4 axioms · 1 invented entities

The load-bearing content rests on standard probability/statistics (conditional expectation as L2 minimizer; residual orthogonality characterizations) plus domain modeling choices for the two numerical systems. No new physical entity is postulated. Free parameters are ordinary ML/experiment knobs (network capacity, likelihood form, Lorenz-96 scale parameters, feature choices for the diagnostic), not constants fitted to force a universal law. The central claim does not require exotic axioms beyond “paired squared loss scores point predictions” and “the scientific task cares about more than the conditional mean.”

free parameters (4)
  • Diagnostic residual feature bank / effect-size thresholds
    Which residual features and what magnitude counts as “large” effect size are modeling choices that control whether the barrier is declared present; they are not uniquely fixed by theory in the paper.
  • Surrogate architecture and training hyperparameters
    Network form, capacity, and optimization settings for deterministic and stochastic surrogates affect residual structure and thus diagnostic outcomes in the numerical studies.
  • Two-scale Lorenz-96 coupling / scale parameters
    Standard Lorenz-96 multi-scale parameters define the closure problem; results on fluctuation suppression are conditional on that setup.
  • Likelihood / stochastic-scale model form
    The “minimal likelihood-based” alternative that recovers variability depends on the chosen parametric conditional family and how it is scored.
axioms (4)
  • standard math Under paired squared loss, the population risk minimizer among measurable predictors is the conditional mean E[Y|X].
    Classical L2 regression fact used to define the barrier and to explain collapse of stochastic outputs under MSE.
  • domain assumption Residual orthogonality to a feature class indicates that the fitted predictor has extracted the L2-projection onto that class; large residual effect size then signals remaining conditional variability rather than simple underfit.
    Operational diagnostic assumption linking statistical residuals to the scientific claim that variability is irreducible given X and the chosen features.
  • domain assumption After coarse graining or partial observation, many CSE prediction tasks are genuinely one-to-many, so the conditional law of Y given X is non-degenerate.
    Problem-setting premise that makes the barrier scientifically relevant rather than a pure math curiosity.
  • domain assumption Task-relevant scientific performance can require matching fluctuation/collective statistics, not only pointwise conditional means.
    Motivates the modeling prescription to score richer features of the conditional law when residual variability matters.
invented entities (1)
  • conditional-mean barrier (named diagnostic concept) no independent evidence
    purpose: Names the failure mode where an MSE surrogate correctly learns E[Y|X] yet misses task-relevant conditional variability in scientific surrogates.
    Terminological/conceptual packaging of a classical L2 fact for SciML practice; not a new physical object. Independent evidence is the diagnostic behavior on the numerical examples, not an external measurement of a new entity.

pith-pipeline@v1.1.0-grok45 · 16214 in / 3360 out tokens · 33892 ms · 2026-07-12T15:49:48.053587+00:00 · methodology

0 comments
read the original abstract

Many prediction tasks in computational science and engineering become one-to-many after coarse graining and partial observation. In such settings, deterministic surrogates trained by squared loss may learn a well-defined mathematical object, the conditional mean, while still missing the task-relevant variability in the underlying conditional law. In this work, we formulate this limitation as the conditional-mean barrier and develop a diagnostic framework for identifying it in fitted scientific machine-learning surrogates. The framework combines residual-feature orthogonality and effect-size diagnostics to distinguish deterministic underfitting from irreducible conditional variability. We also make explicit a simple consequence of paired squared loss: stochastic outputs do not by themselves overcome the barrier, because the objective penalizes model variance and drives the predictor back to the conditional mean. The diagnosis therefore yields a modeling prescription: when residual variability matters, the loss must score richer features of the conditional law rather than a point prediction. Reproducible numerical studies on a controlled two-branch law and a two-scale Lorenz-96 closure problem show how the diagnostic identifies the barrier, how deterministic closures can suppress collective fluctuation statistics in rollout, and how a minimal likelihood-based stochastic-scale model can recover substantially more variability.

Figures

Figures reproduced from arXiv: 2605.28076 by Junfeng Chen.

Figure 1
Figure 1. Figure 1: The conditional-mean barrier. In the deterministic regime (left), increasing model capacity drives the squared-loss risk [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: A controlled two-branch experiment. (a) Samples from (22), the two branches [PITH_FULL_IMAGE:figures/full_fig_p013_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Residual–feature diagnostics for the two-branch example, for the degree-2 (underfit) and degree-9 (high-capacity) least-squares fits, [PITH_FULL_IMAGE:figures/full_fig_p014_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Diagnostic results for the two-scale Lorenz–96 closure. (a) One-step closure [PITH_FULL_IMAGE:figures/full_fig_p016_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Lorenz–96 slow-energy statistics from independent long rollouts of the reference, deterministic-mean, and stochastic closures. (a) Slow [PITH_FULL_IMAGE:figures/full_fig_p018_5.png] view at source ↗

Review history (2 revisions) →

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Intrinsic Instantaneous Coarse-to-Fine Recoverability in the Lorenz-96 System

    nlin.CD 2026-07 accept novelty 6.0

    In Lorenz-96, high-mode instantaneous recoverability from lower modes forms a finite band organized around the triad scale ⌈k/2⌉ and shrinks as forcing increases.

Reference graph

Works this paper leans on

1 extracted references · 1 linked inside Pith · cited by 1 Pith paper

  1. [1]

    ���������� ��� ���������������� ������� �� ��������� ���������������� ���������� ������� ���� ���������� �� ������������ ��� ���� ���� ���������� �� ������� ��� ����������� ���� ����� ����� �������� ���� ���������� ����� �� ������������� ������� ��� ����������� ������ ����������� ����� ������ �������� ��� ������� ������������ �� ���� ��������� �����������...

    Pith/arXiv arXiv 2026