{"id":"2c53152b-f86a-46ca-a5e9-44d154ce796a","arxiv_id":"2412.05806","paper_version":2,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"Regression with noisy input features shrinks predictions toward the mean by a factor 1 divided by (1 plus the squared ratio of noise to signal spread), and this bias persists regardless of training sample size, label accuracy, or sample distribution.","lead":"Machine learning models that turn noisy astronomical spectra into stellar properties systematically shrink their predictions, pulling extreme values toward the average. This paper shows the effect is the statistical phenomenon called attenuation bias and explains exactly when it becomes serious for surveys like APOGEE, LAMOST, and DESI.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The title and abstract generalize attenuation bias to all ML models, but every derivation and simulation is linear; Section 2.3 concedes nonlinear attenuation cannot be predicted, leaving the central claim unsupported for neural networks.","rationale":"I read the paper in good faith. The univariate and multivariate linear-regression derivations are standard attenuation bias, correctly presented, and the synthetic APOGEE spectral experiments with linear models are carefully executed and clearly described. The correlated-feature mitigation result, including the demonstration that temperature scatter weakens the protective correlations, is a genuine and useful contribution. The single load-bearing weakness is the gap between the linear-regression results and the paper's title and abstract, which promise an explanation for why 'machine learning models' systematically underestimate extreme values. The reader's weakest_assumption identifies exactly this gap, and I agree. Section 2.3 is candid that highly nonlinear transformations cannot be analytically predicted or corrected, yet the conclusion asserts that the insights 'naturally extend' to nonlinear models without a derivation or a single nonlinear simulation. This is an external-validity concern, not an internal inconsistency; the linear claims stand on their own. The concrete test I propose would settle it directly: train a neural network and a linear model on the same noisy synthetic spectra and compare their predicted-vs-true slopes. If the neural network attenuates by the same lambda, the broad claim is supported empirically; if it does not, the title and abstract should be narrowed. Because the reader's verdict is CONDITIONAL and my concern does not change that verdict, I set verdict_should_be to UNCHANGED.","tokens_in":36233,"tokens_out":3519,"duration_ms":39794,"concrete_test":"Construct a controlled synthetic experiment: generate APOGEE-like spectra with The Payne at fixed Teff/logg, varying [Fe/H] uniformly in [-1,0], add Gaussian noise at SNR = 100, 30, 10, then train an MLP and a linear regression on the same noisy training set (n=1000) using 1, 10, 100, and 179 pixels. On a noiseless validation set, regress predictions against true [Fe/H] and measure the slope lambda_y for each model and SNR. If the MLP's lambda_y differs by more than 10% from the linear model's lambda_y at the same SNR and pixel count, or is statistically consistent with 1 where the linear model predicts significant attenuation, the overgeneralization to nonlinear ML models is not supported; the paper should then be revised to scope its claims to linear (or weakly nonlinear) models.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 2.1 and 2.4 rigorously derive lambda = 1/(1+(sigma_x/sigma_range)^2) for univariate and multivariate linear regression, and Section 3 validates it on synthetic APOGEE spectra with linear models. That supports the strongest_claim as stated for linear regression. However, the title and abstract claim that 'machine learning models' systematically underestimate extreme values, and the conclusion asserts the insight 'naturally extend[s] to more complex nonlinear models.' No nonlinear model is ever trained in the paper. Section 2.3 explicitly states that for highly nonlinear transformations the bias 'cannot be analytically predicted or corrected,' and its polynomial analysis only reports the attenuation of the highest-order coefficient, not the predicted label, with simulations deviating increasingly from theory as sigma_x grows. Empirically observed compression in neural-network spectroscopic tools (e.g., Leung and Bovy 2019) is not a controlled test of the mechanism; it could stem from label noise, regularization, or model misspecification. Since the central claim is exactly the broad generalization, this is the load-bearing gap: if a neural network trained on noisy inputs does not compress predictions by lambda (or compresses them less, because it can exploit many correlated features), the paper's title claim fails even though the linear result stands.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper derives and simulates attenuation (regression dilution) bias in linear regression with noisy input features. It obtains the formula \\lambda = 1/(1+(\\sigma_x/\\sigma_{\\rm range})^2) for the univariate case, extends the result to independent multivariate features, and gives an analytical result for perfectly correlated features under the assumptions x_{\\rm true,j} = a_j x_{\\rm true} and \\beta = \\beta a. It then applies the framework to synthetic APOGEE-like spectra, showing that even at R=24,000 and SNR=100 the predicted [Fe/H] is compressed by roughly one percent, with stronger effects at lower resolution or SNR. The final sections discuss implications for spectroscopy, distance measurements, and model choice, and argue that the insight extends beyond linear regression to machine learning generally.","tokens_in":36497,"tokens_out":7284,"duration_ms":74633,"significance":"If the linear-regression result is all that is claimed, the paper is a clean and useful contribution: the attenuation formula is derived from explicit assumptions rather than fitted, the simulations reproduce the formula across a wide range of \\sigma_x/\\sigma_{\\rm range}, and the application to spectral inference gives a concrete quantitative warning. The correlated-feature case is also a helpful idealization, though its assumptions are narrow. The main value is as a reference for why linear spectral models compress labels. However, the paper's framing as a general statement about machine learning models is not supported by the evidence; the general claim is the part that would make the paper important beyond the linear case, and it currently rests on extrapolation. With the title and abstract restricted, the contribution is solid but more modest.","major_comments":[{"comment":"The paper's central claim as stated in the title, abstract, and conclusion—that machine learning models systematically underestimate extreme values because of input-feature noise—is broader than what is derived and tested. All analytical results and all numerical experiments in §§2 and 3 are for linear regression (or, in §2.3, for the attenuation of the highest-order coefficient of a polynomial); §2.3 states that for highly nonlinear transformations the bias 'cannot be analytically predicted or corrected.' No neural network or other nonlinear model is trained anywhere in the paper. The empirical citations (e.g., Leung & Bovy 2019) show compression in real neural-network pipelines but cannot isolate the mechanism from label noise, regularization, or model misspecification. The general claim therefore currently rests on an extrapolation. I would ask the author either to restrict the title, abstract, and conclusion to linear and low-order polynomial discriminative models, or to add controlled nonlinear experiments (e.g., a small neural network on the same synthetic spectra) that test whether the compression factor follows \\lambda = 1/(1+(\\sigma_x/\\sigma_{\\rm range})^2).","section":"Title, Abstract, §§3–5"},{"comment":"The polynomial analysis defines \\lambda_{\\beta,n} as the attenuation of the highest-order regression coefficient, not as the attenuation of the predicted label \\lambda_y. For linear regression these two coincide, but for polynomial features they need not: even if the fitted coefficient \\hat{\\beta}_n is attenuated by \\lambda_{\\beta,n}, the prediction y_{\\rm pred} = \\hat{\\beta}_n x_{\\rm obs}^n is affected both by the coefficient attenuation and by the noise in x_{\\rm obs}^n itself. The simulations in Fig. 2 only report \\lambda_\\beta, and they show growing deviations from the approximate formula as \\sigma_x increases. Since the paper's subject is compression of predicted values, the discussion in §2.3 and the conclusions drawn from it (including the claim that higher-order terms suffer more severe bias) should be supported by direct measurements of \\lambda_y for polynomial models, or explicitly restricted to coefficient attenuation.","section":"§2.3, Eq. (22), Fig. 2"},{"comment":"The multivariate correlated-feature result—that attenuation is mitigated as p grows—is derived under the strong assumptions that all features are perfectly correlated as x_{\\rm true,j} = a_j x_{\\rm true} and that the true coefficients are proportional to the scaling factors, \\beta = \\beta a, in addition to the first-order expectation approximation in Eq. (27). The author acknowledges these assumptions, but the numerical validation in Fig. 5 uses exactly the same idealized construction, so it does not test the robustness of the p\\to\\infty mitigation claim outside this special case. In the APOGEE application (Fig. 8), the theoretical curves are replaced by simulations, and the paper notes that real parameter variations weaken correlations (Fig. 9). I would recommend adding at least one simulation with imperfect correlation or with \\beta not aligned with a, to show whether the qualitative conclusion—more correlated features reduce bias—survives moderate violation of the alignment assumption.","section":"§2.4.2, Eq. (43), Appendix C"}],"minor_comments":[{"comment":"The word 'substracted' appears where 'subtracted' is intended; this typo occurs in at least two places.","section":"§2.1, §2.4"},{"comment":"In the sentence beginning 'The systematic underestimation of distances due to attenuation bias', the word 'whic' should be 'which'.","section":"§4.2"},{"comment":"The phrase 'how attenuation bias scales also applies to nonlinear transformations' is grammatically awkward and should be rephrased for clarity.","section":"§2.3"},{"comment":"Several references are incomplete: Fuller (1987), Carroll et al. (1995), and a few other entries lack titles and publishers; the bibliography should be made consistent.","section":"References"},{"comment":"The top axis is labeled 'SNR' but the text specifies that all SNR values are per pixel; the axis label or caption should state this convention explicitly.","section":"Fig. 8"},{"comment":"The figure in Appendix D is called 'Figure C1' in the text, which is inconsistent with its placement; the labeling should be corrected.","section":"Appendix D"}],"recommendation":"major_revision","confidential_remarks":"To the editor: the core linear-regression material is sound and likely useful to the astro-ML community, but the paper is currently over-sold. If the author is unwilling to add nonlinear simulations, I would recommend that the title and abstract be revised to explicitly limit the claim to linear and low-order polynomial models. The self-citation of The Payne emulator is not a concern because the emulator's accuracy is not load-bearing. The paper may also benefit from a note that the univariate attenuation formula is a classical result (Fuller 1987); the new contributions are the correlated multivariate treatment and the astronomical application."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague —\n\nRead this before you next calibrate a spectroscopic label transfer, because it's the clearest writeup of attenuation bias in the astro-ML context. For linear regression, the central result is solid: with input noise sigma_x on a feature of intrinsic spread sigma_range, the fitted coefficients and therefore the predictions shrink by lambda = 1/(1+(sigma_x/sigma_range)^2), independent of sample size and label noise. Their synthetic simulations match the derivation, and the APOGEE-like demonstration—percent-level bias at SNR 100, much worse at LAMOST/DESI SNR—is a practical warning that should be taken seriously. The correlated-feature extension, Eq. 43, is a real addition, and the polynomial-order scaling Eq. 22 is useful intuition. Credit where due: the paper is careful about its assumptions, provides appendices, and does not oversell the accuracy of the Payne emulator; that part isn't load-bearing.\n\nThe soft spot is exactly where the title points. The paper proves things for linear regression—and for polynomial features in a linear model—but claims in the title, abstract, and conclusion that machine learning models systematically underestimate extreme values. Section 2.3 explicitly concedes that for highly nonlinear transformations the attenuation 'cannot be analytically predicted or corrected,' and no neural network or other nonlinear model is ever trained. So the broad generalization is unsupported. It may be true, and the 'generalized linear regression' framing is a reasonable heuristic, but the paper doesn't demonstrate it. That's a disconnect between evidence and marketing that a referee should flag. It's fixable by softening the title or adding a nonlinear simulation; it doesn't invalidate the linear results.\n\nA second, lesser caveat: the correlated-feature attenuation formula assumes perfect correlation and beta proportional to a, which the paper acknowledges. The simulations show the mitigation weakens when temperature scatter decorrelates pixels. So the 'correlation saves you' message is best-case; real spectra are messier, as the paper itself demonstrates.\n\nVerdict: send it to review with the expectation of moderate revision. The core is sound and useful; the overclaim needs attention. I'd bring it to a reading group focused on survey science tools.","headline":"A solid linear-regression treatment of attenuation bias for astronomy, embedded in a title that overclaims to all ML; worth refereeing with revision.","tokens_in":36948,"tokens_out":2644,"would_cite":true,"duration_ms":28239,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Measurement noise in input features alone compresses machine-learning predictions toward the mean, no matter how much training data is used.","keywords":["attenuation bias","regression dilution","machine learning","stellar spectroscopy","measurement uncertainties","spectroscopic surveys","data-driven models"],"falsifier":"Train any nonlinear regressor, such as a three-layer network, on synthetic data with known noise σ_x and measure the slope of predictions against true labels over a grid of σ_x/σ_range; if that slope does not follow λ = 1/(1+(σ_x/σ_range)^2), the paper's generalization to ML models fails, even though its linear result may stand.","tokens_in":36039,"feed_emoji":"📉","tokens_out":4497,"duration_ms":41723,"temperature":0.7,"pith_summary":"The paper argues that the well-known tendency of astronomical machine learning models to compress the dynamic range of their predictions—high values predicted too low, low values predicted too high—is not a training failure but a statistical inevitability. In linear regression, any measurement error in the input features attenuates the fitted coefficients by the factor λ = 1/(1+(σ_x/σ_range)^2), where σ_x is the input noise and σ_range is the true spread of the signal. This bias survives unlimited training data, perfect labels, and balanced sampling. The author shows the same attenuation in independent multivariate regression, demonstrates that correlated features (as in real spectra) reduce but do not eliminate it, and finds percent-level bias even for APOGEE-quality spectra at R=24,000 and SNR=100. The paper concludes that generative models that forward-model the observables are structurally safer than discriminative models for such inference.","feed_headline":"Noisy input pixels alone shrink ML predictions toward the mean","feed_subtitle":"Proof shows the bias survives any training set size, so spectroscopy surveys must model input errors.","key_machinery":"The load-bearing object is the attenuation factor λ = 1/(1+(σ_x/σ_range)^2), derived from the ordinary least squares expectation E[β̂] = Cov(x_obs,y_obs)/Var(x_obs). Input noise inflates the denominator without changing the numerator, so every coefficient is shrunk toward zero by the same ratio, and the predicted label is multiplied by λ. The multivariate extensions use the same decomposition: independent features give each coefficient its own λ_j, while perfectly correlated features produce a collective factor λ = (Σ a_j²)/(nσ_x²/S_x + Σ a_j²) that improves with dimensionality. For polynomial features the attenuation roughly scales as λ_n = 1/(1+n²(σ_x/σ_range)²), so higher-order terms suffer more severely.","core_discovery":"The central claim is that attenuation bias in direct spectra-to-label regression is a fundamental consequence of input measurement error, not of limited samples or imperfect labels. For univariate linear regression the expected slope is E[β̂] = λβ with λ = 1/(1+(σ_x/σ_range)^2), so predictions on new data are compressed by the same factor λ. The paper proves this bias is independent of training sample size and label noise, and shows it persists for independent multivariate features regardless of dimension. For perfectly correlated features the attenuation factor grows less severe with the number of features, approaching unity in the infinite-dimension limit, but real spectra fall short of that ideal because parameter variations such as temperature decorrelate features. Applied to APOGEE mock spectra, the framework predicts 1% bias under optimistic conditions and far larger biases at LAMOST/DESI-like resolutions and SNRs, with especially strong attenuation for elements that have few spectral features.","pith_inferences":["If the linear-regression result carries to nonlinear networks, then any error-ignorant discriminative model trained on noisy inputs will compress predicted labels even when its training loss is minimized; a direct test would compare the predicted-versus-true slope of a small neural network to λ across σ_x.","The bias may explain part of the label-transfer discrepancies seen when stellar parameters are transferred between surveys; much of the residual in such catalogs could be modeled with the λ formula rather than treated as zero-point scatter.","Because the bias is set by a ratio, co-adding or binning pixels (thereby reducing σ_x) would shrink attenuation even without changing the model, offering a cheaper mitigation than new instruments.","The generative-versus-discriminative asymmetry suggests that hybrid approaches that fit spectra forward, or models that explicitly propagate input noise, will become necessary as surveys push to lower SNR."],"forward_implications":["More training data or more precise labels cannot fix the bias; only reducing input measurement uncertainty or exploiting feature correlations helps.","In high-resolution, high-SNR spectroscopy (APOGEE-like), the bias is at the percent level, enough to matter for abundance and distance work; low-resolution surveys suffer much worse.","Elements with few clean spectral features, such as oxygen, potassium, and vanadium, are especially affected because they cannot benefit from the correlated-feature mitigation.","Discriminative models inherit input-noise bias, whereas generative models place the noise in the dependent variable and are therefore not attenuated by this mechanism.","Power-law relations studied in log-log space, such as the M-σ relation or the Tully-Fisher relation, are vulnerable whenever dynamical-range-to-error ratios are O(1)-O(10)."],"supporting_citations":[{"why":"Establishes the errors-in-variables framework and the classical attenuation-bias formula that the paper builds on.","marker":"Fuller (1987)"},{"why":"Provides the nonlinear measurement-error models that motivate the paper's discussion of polynomial and machine-learning extensions.","marker":"Carroll et al. (1995)"},{"why":"Supplies an astronomical treatment of measurement errors in regression that the paper contrasts with and extends.","marker":"Kelly (2007)"},{"why":"Gives the accessible review of regression dilution that underpins the paper's presentation of the classical result.","marker":"Frost and Thompson (2002)"},{"why":"Provides The Payne spectral emulator used to generate the APOGEE mock spectra for the case study.","marker":"Ting et al. (2019)"},{"why":"Introduces The Cannon, the linear/polynomial spectral model cited as a practical example where the framework applies directly.","marker":"Ness et al. (2015)"},{"why":"Shows empirically observed attenuation in neural-network stellar parameter predictions that the paper's framework aims to explain.","marker":"Leung and Bovy (2019)"}],"fun_headline_variants":["Input noise alone shrinks ML predictions toward the mean","Measurement error, not sample size, drives ML prediction bias","Spectroscopy ML bias: input errors compress predictions","Why ML underestimates extremes: input noise is the culprit","Proof: input noise compresses ML predictions, independent of data size"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The paper's broad title claims apply to all machine learning models, but its analytical proof covers linear regression; the step that generalizes to nonlinear networks is asserted, not derived.","fun_headline_variants_meta":{"raw":{"variants":["Input noise alone shrinks ML predictions toward the mean","Measurement error, not sample size, drives ML prediction bias","Spectroscopy ML bias: input errors compress predictions","Why ML underestimates extremes: input noise is the culprit","Proof: input noise compresses ML predictions, independent of data size"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001011,"raw_usage":{"total_tokens":4273,"prompt_tokens":946,"completion_tokens":3327,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":562,"completion_tokens_details":{"reasoning_tokens":3246}},"tokens_in":562,"tokens_out":3327,"duration_ms":21685,"temperature":1.0,"reasoning_tokens":3246,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T20:20:21.673704+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train any nonlinear regressor, such as a three-layer network, on synthetic data with known noise σ_x and measure the slope of predictions against true labels over a grid of σ_x/σ_range; if that slope does not follow λ = 1/(1+(σ_x/σ_range)^2), the paper's generalization to ML models fails, even though its linear result may stand.","supporting_citations":[],"review_version":1}