{"id":"2804e076-4231-4632-b2d0-dad5b9af3965","arxiv_id":"2411.12060","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"A Taylor-expansion method derives feature coefficients from a compressing feature and compares them with ridge and PLS regularization paths, revealing how coefficients move from variance-dominated to nonlinearity-approximating shapes.","lead":"Linear regression on high-dimensional functional data is usually hard to interpret, and this paper proposes a way to see what the coefficients are doing by comparing them with the linearized version of a known nonlinear feature. The method is illustrated on battery voltage curves, where it shows that strongly regularized coefficients first capture high-variance directions and then start approximating nonlinear structure as regularization is relaxed.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The L2-distance comparison in the p≫n regime conflates the row-space projection of βT1 with βT1 itself; the claimed coefficient similarity may be a projection artifact.","rationale":"The paper's derivations are internally consistent, the synthetic case studies are clearly described, and the automatic-differentiation implementation is a practical strength. The reader's Assumptions 1–2 concern is legitimate and is partially acknowledged by the authors in the Conclusion, but the most immediately testable gap for the central claim is geometric: the comparison uses a full-space L2 distance when all regression coefficients are confined to a low-dimensional row space. Because the regularization parameter is selected by minimizing that same distance, the apparent agreement between β(λ*) and βT1 could be driven by the projection of βT1 onto the row space, independent of how close βT1 is to the row space or how prediction-relevant the residual is. The proposed decomposition and prediction-space comparison would settle this without changing the paper's mathematical core. I therefore keep the reader's conditional verdict: the method is sound, but the central similarity claim needs this qualification or an explicit projection-based restatement.","tokens_in":7100,"tokens_out":9265,"duration_ms":104756,"concrete_test":"For each case study, compute the mean-centered data matrix X_c, the row-space projector P = X_cᵀ(X_c X_cᵀ)⁺X_c, and report ||βT1||², ||PβT1||², ||(I−P)βT1||², and the minimizing λ from (10); then recompute the comparison using the prediction-space distance (13) and compare the resulting λ and coefficient plots. If the nullspace component is substantial or if the prediction-space λ* differs materially, the similarity claim should be restated as being about the row-space projection of βT1 rather than about βT1 itself.","verdict_should_be":"UNCHANGED","load_bearing_attack":"With n=40 and p=1000, every ridge and PLS coefficient vector lies in the 39-dimensional row space of the mean-centered data, while the feature coefficient βT1=m∇g(x̄) generally has a component in the 969-dimensional nullspace. For the orthogonal projector P onto the row space, ||β(λ)−βT1||² = ||β(λ)−PβT1||² + ||(I−P)βT1||². The second term is constant in λ, so minimizing (10) only fits the row-space projection PβT1; the nullspace component is unfixable by regularization and is not reflected in the shape comparisons made in the figures. This matters because the central claim is that the regularization path 'contains regression coefficients similar to the linearized true mapping,' not merely that it approaches a projection of that mapping. Section A.3 acknowledges that nullspace differences can be prediction-irrelevant, but the paper never quantifies the decomposition for the two case studies, so the similarity statement is not yet pinned down in the full coefficient space.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies how linear regression coefficients for high-dimensional functional data relate to a known nonlinear compressing feature. The authors linearize a feature g at the training mean, define feature coefficients β_T1 = m∇g(x̄) with m estimated by OLS, and then select ridge and PLS regularization parameters by minimizing the ℓ2 distance between the regression coefficients and β_T1. They illustrate the method on two synthetic battery-data responses, one based on a sum-of-squares feature and one on a sinusoidal feature, and conclude that strongly regularized regression coefficients resemble the linearized feature coefficients, while weaker regularization produces coefficients that exploit local structure to approximate nonlinearity.","tokens_in":7335,"tokens_out":6379,"duration_ms":70690,"significance":"If the central claim holds, the paper offers an interpretability link between nonlinear feature maps and linear regression coefficients in the p ≫ n regime, with potential relevance to battery cycle-life prediction. The manuscript is commendably explicit about its assumptions and limitations, and the use of automatic differentiation for deriving feature coefficients is a practical strength. However, the empirical evidence is limited to two synthetic responses from a single training set with no uncertainty quantification or out-of-sample validation, and the comparison metric is partly constructed to make the closest coefficients close. The theoretical derivation in Section 2 is sound, but the load-bearing empirical conclusions in Section 4 need additional support before the claims can be accepted as stated.","major_comments":[{"comment":"The 'closeness' of the regression path to β_T1 is enforced by construction: the paper minimizes the ℓ2 distance to define the closest regression coefficient and then presents that coefficient as evidence that the path 'contains' similar coefficients. To support the claim, the paper should report the minimized distance values, compare them with distances at other regularization parameters (e.g., cross-validation-selected values), and state how much the visual similarity in Figures 2 and 3 exceeds what would be expected under a null model or under random row-space vectors.","section":"Section 2, Eqs. (9)-(10)"},{"comment":"In the p ≫ n setting with n = 40 and p = 1000, every ridge and PLS coefficient vector lies in the 39-dimensional row space of the mean-centered data, while β_T1 generally has a component in the 969-dimensional nullspace. The ℓ2 distance decomposes as ||β(λ) − β_T1||² = ||β(λ) − Pβ_T1||² + ||(I − P)β_T1||², where P projects onto the row space. The nullspace term is independent of λ, so minimizing Eq. (10) fits only Pβ_T1. The paper acknowledges nullspace effects in A.3 but never quantifies the decomposition for the two case studies. Without reporting ||Pβ_T1|| and ||(I − P)β_T1||, the visual shape comparisons in Figures 2 and 3 do not establish similarity in the full coefficient space; they may only show that the row-space projection resembles β_T1.","section":"Section 2, A.3, Figures 2-3"},{"comment":"The central empirical claim is supported by only two synthetic responses, evaluated on the same training set used both to estimate the scalar m in Eq. (7) and to select the regularization parameter in Eq. (10). There is no uncertainty quantification, no out-of-sample validation, and no quantitative measure of shape agreement other than the distance that was minimized by construction. The conclusions in Section 4 generalize from these two examples; the authors should either add bootstrap confidence bands, repeat the analysis across a broader family of features, or explicitly restrict the conclusions to illustrative demonstrations.","section":"Section 3, Figures 2-3"},{"comment":"The feature coefficients β_T1 are not fully specified by the feature g alone, because the scalar m is obtained by OLS on the same response vector y that is later used in the regression and in the distance minimization of Eq. (10). Thus the target of the comparison already contains information from y, which weakens the interpretation of β_T1 as the 'linearized true mapping.' The paper should report how sensitive the conclusions are to this calibration, for instance by comparing with m fixed at 1 or by estimating m on a separate validation split.","section":"Section 2, Eqs. (7)-(8)"}],"minor_comments":[{"comment":"The paper states X ∈ R^{124×1000} in Section 3 but R^{123×1000} in Appendix A.1 after outlier removal; please clarify the intended dimensions.","section":"Section 3 vs. A.1"},{"comment":"The text says Assumption 2 (zero irreducible error) is relaxed later, but the only mechanism shown is the OLS scalar m in Eq. (7); a single scalar cannot absorb per-sample irreducible errors ε_i^*. Please state explicitly how Assumption 2 is relaxed, or remove the claim.","section":"Section 2, Assumption 2"},{"comment":"No numerical details are given for solving the optimization in Eq. (10), such as the grid or algorithm for λ, the range of λ values, or how PLS component counts were searched. These details are needed for reproducibility.","section":"Section 2, Eqs. (9)-(10)"},{"comment":"The sentence 'Systemically designing features' contains a typo and should read 'Systematically designing features'; similar minor typographical issues appear elsewhere, e.g., 'V oltage' in the Introduction and 'T est' in Figure 1.","section":"Section 4"},{"comment":"The manuscript refers to 'the software accompanying the article' but does not provide a repository or DOI; please include the URL or a data/code availability statement.","section":"Section A.2"}],"recommendation":"major_revision","confidential_remarks":"To the editor: This short communication builds directly on the authors' earlier work in [8], and the incremental contribution is mainly the linearized-feature comparison and the two synthetic case studies. The derivations are sound, but the central empirical claim is not yet fully supported because of the row-space projection issue and the lack of uncertainty quantification. I would not recommend rejection if the authors can quantify the nullspace decomposition and temper the general conclusions, but the manuscript in its current form needs substantive revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear [Name],\n\nHere's my take on 2411.12060. The paper is a short communication that linearizes a nonlinear compressing feature g to get feature coefficients β_T1 = m∇g(x̄), then compares them to ridge/PLS coefficient paths via ℓ2 distance on two synthetic battery case studies. What's actually new is the comparison idea and the two case studies; the Taylor expansion and OLS steps are standard. The derivation is correct, the figures are clear, and the authors are honest that the responses are synthetic and no ideal feature exists in practice. They also acknowledge in A.3 that nullspace can make coefficient distances large while predictions agree. That's more candor than most.\n\nThe soft spots are real but addressable. First, the stress-test point is correct: for n=40, p=1000, every ridge and PLS coefficient lives in the 39-dimensional row space, while β_T1 generally has a nullspace component. Minimizing ||β(λ)-β_T1||² only fits the row-space projection; the nullspace term is constant in λ. So the \"closest\" coefficient is closest to a projection of β_T1, not to β_T1 itself, and the figures don't decompose the two. The paper mentions this possibility but never quantifies it, so the central claim that the path \"contains regression coefficients similar to the linearized true mapping\" is not yet established in full coefficient space. This doesn't break the method, but it needs either a decomposition or a softer claim.\n\nSecond, there's a mild circularity: m is fitted by OLS to the same y used in the regression, and λ is chosen to minimize distance to β_T1. That's not fatal—it's a descriptive tool, not a predictive claim—but it means the \"closeness\" is partly constructed. The strongest interpretive statements in the conclusion (\"linear models can perform well\") go beyond what two synthetic responses can support, especially with no uncertainty quantification or out-of-sample validation.\n\nOverall, this is a solid little paper with a correct core and an overreaching headline. A serious referee should engage with it; the issues are fixable with a decomposition of the nullspace projection and more careful wording. I'd bring it to a reading group if someone is working on high-dimensional coefficient interpretation, but I wouldn't cite it as evidence that regression paths approximate feature coefficients until the projection issue is addressed.\n\nNet: worth reviewing, needs revision.","headline":"Useful, well-scoped short paper with a correct derivation, but the headline similarity claim is undercut by a nullspace projection issue the paper never quantifies.","tokens_in":7808,"tokens_out":2567,"would_cite":false,"duration_ms":25985,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper shows that ridge and partial least squares regression paths in high dimensions contain coefficients that, under strong regularization, match the linearization of a single nonlinear compressing feature, and that relaxing…","keywords":["linear regression","high-dimensional data","feature coefficients","compressing features","regularization path","ridge regression","partial least squares","lithium-ion batteries"],"falsifier":"Take the same battery data but build a response from a non-differentiable feature, for example $g(x_i)=\\sum_j \\mathbf{1}_{x_{i,j}>c}$, or from a sum of two compressing features. If the ridge path still passes close to the $\\beta_{\\mathrm{T1}}$ of some differentiable single feature at strong regularization, or if cross-validated coefficients do not vary faster than strongly regularized ones, the claimed interpretation would need to be revised.","tokens_in":6934,"feed_emoji":"🔋","tokens_out":7779,"duration_ms":76191,"temperature":0.7,"pith_summary":"Linear regression in high dimensions is hard to interpret because many coefficient vectors make the same predictions. This paper develops a reference for reading regression coefficients: take a single nonlinear compressing feature $g$ that generates the response, linearize it around the data mean, and rescale to obtain feature coefficients $\\beta_{\\mathrm{T1}} = m\\nabla g(\\bar{x})$. Using ridge regression and partial least squares on lithium-ion battery data with synthetic responses, the paper shows that strongly regularized coefficients align with these feature coefficients and are dominated by data variance, while coefficients chosen by cross-validation grow in magnitude and vary faster across the input domain, linearly approximating the nonlinear response locally. The work's aim is to explain how the shape of high-dimensional regression coefficients changes with regularization and to connect regression paths to interpretable feature engineering.","feed_headline":"Ridge coefficients match a linearized feature at strong regularization","feed_subtitle":"Taylor-expanding the true feature explains how linear models absorb nonlinearity in high dimensions.","key_machinery":"The load-bearing object is the vector of feature coefficients $\\beta_{\\mathrm{T1}} = m\\nabla g(\\bar{x})$, where $g:\\mathbb{R}^p\\to\\mathbb{R}$ is a single compressing feature, $\\bar{x}$ is the column mean of the data, and $m$ is a scalar estimated by ordinary least squares after centering. This vector is the first-order Taylor approximation of $g$ rescaled to match the response. It supplies a reference against which the ridge and PLS solution paths are compared using the $\\ell^2$ distance, and it is what lets the authors separate the part of the coefficients explained by the linearized feature from the part that emerges as regularization is relaxed.","core_discovery":"The central claim is that the ridge regression solution path contains coefficients that, at strong regularization, are comparable to the first-order linearization of the true scalar feature: $\\beta_{\\mathrm{T1}} = m\\nabla g(\\bar{x})$, where $m$ is an ordinary-least-squares scalar fit. In the sum-of-squares case study these coefficients are a scaled column mean; in the sinusoidal case study they take a different shape that does not match the variance of the data. The closest ridge and PLS coefficients, selected by minimizing $\\|\\beta(\\lambda)-\\beta_{\\mathrm{T1}}\\|_2^2$, track this shape, and under cross-validated regularization the coefficients show larger amplitude and faster variation across voltage, indicating the linear model is using local structure to approximate a nonlinear map. This gives a concrete picture: regularized linear regression first captures high-variance directions, then spends its remaining capacity on a local, piecewise-linear approximation of the nonlinearity.","pith_inferences":["Editorial extension: If this picture generalizes, the gap between $\\beta_{\\mathrm{T1}}$ and the cross-validated coefficients is a measurable 'nonlinearity load' of the data set, which could be tracked across different candidate features.","Editorial extension: The nullspace caveat noted in the paper means closeness of coefficients is a stricter condition than closeness of predictions; a practitioner could use prediction distance instead when the goal is forecasting, but would lose the shape interpretation.","Editorial extension: For real battery data where no ideal feature is known, the method suggests a practical test: propose a feature, linearize it, and check whether the ridge path comes near it; if it does, that feature explains the regression coefficients even when it was not used to build the response."],"forward_implications":["If a response is generated by a single differentiable compressing feature, ridge coefficients at strong regularization will look like the scaled gradient of that feature at the data mean, not like an arbitrary regularization artifact.","At cross-validation-level regularization, coefficient magnitude and the rate of change across the input domain increase, so the shape of the coefficients can be read as a local, linear approximation of the nonlinear response.","The $\\ell^2$ distance to $\\beta_{\\mathrm{T1}}$ along the regularization path gives a principled way to pick the regularization level at which a linear model best matches a candidate feature linearization.","PLS with one component mirrors ridge in the variance-dominated regime, but may need additional components before it picks up the characteristic shape of the feature coefficients."],"supporting_citations":[{"why":"Supplies the lithium-ion battery data set and the variance compressing feature that motivate the case studies and the synthetic responses.","marker":"[12]"},{"why":"Establishes the nullspace and regularization view of high-dimensional regression coefficients that the comparison with feature coefficients builds on.","marker":"[8]"},{"why":"Gives the ridge regression path formulation, ordinary least squares background, and the one-standard-error rule used to select cross-validated regularization.","marker":"[4]"},{"why":"Shows that linear regression on the raw high-dimensional battery data performs comparably to the variance feature, motivating the coefficient comparison.","marker":"[11]"},{"why":"Supports the claim that linear models work well on high-dimensional data when nonlinearity is limited.","marker":"[6]"},{"why":"Supplies the repeated double cross-validation scheme behind the regularization choices compared in the case studies.","marker":"[15]"}],"fun_headline_variants":["Linear regression's ridge path mirrors linearized nonlinear features","How ridge regression approximates nonlinear features via linearization","Strong-regularization ridge coefficients match a Taylor-expanded feature","Ridge path at high regularization reveals linearized feature shapes","Linearized features underpin ridge coefficient patterns in battery data"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole comparison rests on the assumption that there exists one smooth scalar feature of the inputs that produces the response, with no noise that the linearization cannot absorb; if the true response is generated by multiple features or jumps, the reference coefficients do not represent anything real.","fun_headline_variants_meta":{"raw":{"variants":["Linear regression's ridge path mirrors linearized nonlinear features","How ridge regression approximates nonlinear features via linearization","Strong-regularization ridge coefficients match a Taylor-expanded feature","Ridge path at high regularization reveals linearized feature shapes","Linearized features underpin ridge coefficient patterns in battery data"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000602,"raw_usage":{"total_tokens":2788,"prompt_tokens":897,"completion_tokens":1891,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":513,"completion_tokens_details":{"reasoning_tokens":1811}},"tokens_in":513,"tokens_out":1891,"duration_ms":14386,"temperature":1.0,"reasoning_tokens":1811,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T17:56:45.664374+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take the same battery data but build a response from a non-differentiable feature, for example $g(x_i)=\\sum_j \\mathbf{1}_{x_{i,j}>c}$, or from a sum of two compressing features. If the ridge path still passes close to the $\\beta_{\\mathrm{T1}}$ of some differentiable single feature at strong regularization, or if cross-validated coefficients do not vary faster than strongly regularized ones, the claimed interpretation would need to be revised.","supporting_citations":[{"cited_title":"Severson, Peter M","cited_arxiv_id":null,"evidence_quote":"Supplies the lithium-ion battery data set and the variance compressing feature that motivate the case studies and the synthetic responses."},{"cited_title":"Chueh, Martin Z","cited_arxiv_id":null,"evidence_quote":"Establishes the nullspace and regularization view of high-dimensional regression coefficients that the comparison with feature coefficients builds on."},{"cited_title":"Friedman, and Jerome H","cited_arxiv_id":null,"evidence_quote":"Gives the ridge regression path formulation, ordinary least squares background, and the one-standard-error rule used to select cross-validated regularization."},{"cited_title":"Asinger, Robin Droop, Rolf Findeisen, and Richard D","cited_arxiv_id":null,"evidence_quote":"Shows that linear regression on the raw high-dimensional battery data performs comparably to the variance feature, motivating the coefficient comparison."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supports the claim that linear models work well on high-dimensional data when nonlinearity is limited."},{"cited_title":"Repeated double cross validation","cited_arxiv_id":null,"evidence_quote":"Supplies the repeated double cross-validation scheme behind the regularization choices compared in the case studies."}],"review_version":1}