Robust Non-Linear Correlations via Polynomial Regression

Luca Giuliani; Michele Lombardi

REVIEW 4 major objections 2 minor 14 references

Robust Non-Linear Correlations via Polynomial Regression

T0 review · 4 major / 2 minor · reviewed 2026-08-04 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper introduces HGR-KB and HGR-SK, polynomial-kernel estimators for the HGR maximal correlation that are deterministic, differentiable, and robust, and argues they outperform neural, kernel-density, and randomized baselines.

desk verdict The polynomial-kernel HGR estimator is built on a sign error: Eqs. (8)–(9) maximize a squared residual, making HGR-KB unbounded and unrelated to the HGR objective. read the letter →

arxiv 2509.09380 v1 pith:QXAA6HDG submitted 2025-09-11 cs.LG cs.AIcs.NAmath.NA

Luca Giuliani , Michele Lombardi This is my paper · ORCID

classification cs.LGcs.AIcs.NAmath.NA MSC 62H2062-08

keywords Hirschfeld-Gebelein-Renyicoefficientpolynomialregressionmaximalcorrelationcopulatransformationdependencemeasurefairnessregularizerdifferentiableestimatorleastsquares

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper is trying to make the HGR correlation coefficient practical. HGR measures arbitrary nonlinear dependence between two variables, but its exact value is uncomputable and existing differentiable estimators are stochastic or opaque. The authors claim that representing the two copula transformations as polynomial kernels converts HGR estimation into a least-squares problem with a single variance constraint, solvable exactly and deterministically. If true, practitioners get a fast, interpretable correlation score that also supplies a subgradient usable for fairness constraints in neural-network training.

What carries the argument

The Pearson-as-least-squares reformulation: maximizing a sample correlation is equivalent to minimizing the squared error of the best linear fit after standardizing, and replacing unknown copula maps with degree-d polynomial kernels turns HGR into a constrained quadratic optimization. The paper uses the scale symmetry of correlation to merge one coefficient vector with the regression slope, leaving a single nonconvex variance constraint, and solves the full problem with a trust-region method; the single-kernel variant (degree one in one variable) becomes an ordinary least-squares solve.

What would settle it

On synthetic data with a known non-invertible relationship such as y = x^2, run HGR-KB with degree 2 and compare against the squared Pearson correlation of the true optimal features; if the result differs, the claimed equivalence to maximal correlation fails. A second probe: solve the same instance from multiple trust-region starting points; any dependence on initialization would indicate local optima.

Watch

Extended reading notes

Core claim

The central claim is that finite-degree polynomial expansions of the two copula functions are enough to approximate HGR reliably. By rewriting Pearson correlation as the optimal slope of a least-squares fit between standardized variables, the maximization over functions becomes a quadratic problem in polynomial coefficients with one variance constraint; a single-kernel restriction reduces further to unconstrained least squares. The paper argues the resulting estimators, HGR-KB and HGR-SK, are deterministic, interpretable, differentiable, and more stable than neural, kernel-density, and randomized baselines, with the degrees acting as a transparent bias-variance dial.

Load-bearing premise

The load-bearing premise is that the bi-level HGR problem can be collapsed to a single convex optimization; Appendix B's collapse requires all transformations be invertible, a condition polynomial kernels of degree greater than one violate, and Section C's convex Lagrangian argument applies to a nonconvex objective—so if these fail, HGR-KB computes only a local polynomial optimum.

Editorial extensions

If this is right

HGR-KB yields fully deterministic correlation values, removing seed-dependence of neural and randomized estimators.
Kernel coefficients can be inspected directly, so users can see which polynomial degrees carry the dependence.
The computed value rises monotonically with kernel degrees, giving a principled way to balance underfitting against overfitting.
A subgradient of HGR-KB, and an exact gradient of HGR-SK, can penalize dependence in fair-machine-learning training.
HGR-SK is orders of magnitude faster but only detects functional dependencies, missing relations such as circles.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Because Appendix B's single-level equivalence assumes all transformations are invertible, and polynomials of degree greater than one are not globally invertible, the equality between the computed optimum and the true HGR maximum is not established for the general case; the paper's experimental miss on y = sin(x^2) hints at this gap.
The same least-squares machinery would extend naturally to other basis families (splines, trigonometric polynomials) preserving determinism; the paper does not explore this.
Monotonicity with kernel degree suggests a calibration heuristic: increase degree until the correlation stops improving, though Figure 4 shows the plateau is dataset-dependent.
In the fairness experiments, constraint satisfaction is measured only with the penalizer's own HGR type; the paper states no oracle exists, so cross-method constraint comparisons are semantic rather than direct.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Hi [Colleague],

Quick take on arXiv:2509.09380. The paper is not ready for serious engagement as written. The central estimator HGR-KB is defined in Eqs. (8)–(9) as the arg max of a squared residual subject to a variance constraint. That is the wrong direction: the least-squares formulation of Pearson correlation is a minimization, and maximizing that same objective is unbounded (α is free) and has no connection to HGR. This is not a minor typo; it means the main methodological contribution is mathematically undefined. The stress-test note is right.

What's good: the underlying idea of using polynomial kernels for HGR is sound, and the single-kernel subcase (HGR-SK) is, once corrected to arg min, simply the square root of R² from polynomial regression — fast, deterministic, interpretable. The experiments are thorough in comparing against HGR-NN, HGR-KDE, and RDC, and the determinism advantage is real. The monotonicity result in Appendix D is correct.

Where it falls apart: (1) the sign error in Eqs. (8)–(9) kills the estimator; (2) Appendix C's convexity claim is wrong — the penalty |σ(β)^2−1| is not convex, and Lagrange multipliers don't fix the max/min mismatch; (3) Appendix B's invertibility assumption is irrelevant, and the equivalence doesn't need it; (4) the differentiation mechanism is hand-waved — you can't just say Eq. (7) is differentiable without a subgradient derivation through the arg max (or arg min); (5) the novelty is overstated — polynomial kernel CCA is in the cited literature.

The paper needs major revision before it can be taken seriously. I would not cite it or bring it to our reading group until the authors fix the optimization direction and provide a coherent derivation. If they do, the HGR-SK variant could be a useful practical tool. But as it stands, I'd desk reject it in its current form.

Referee Report

4 major / 2 minor

Summary. The paper proposes two polynomial-kernel estimators of the Hirschfeld–Gebelein–Rényi (HGR) correlation coefficient: HGR-KB, which maximizes the Pearson correlation between polynomial transformations of the two variables, and HGR-SK, a restricted version in which one of the two transformations is required to be linear. The authors derive a least-squares reformulation for HGR-KB, claim a convex Lagrangian formulation with a globally optimal solution, prove monotonicity of HGR-KB in the polynomial degrees, and report experiments on synthetic correlation detection and fairness-constrained machine learning. The general direction is interesting and the intended estimator is potentially useful, but the formal development as written contains major flaws: the central optimization problem is stated as an arg max of a squared residual, which is both the wrong direction for least-squares correlation and an unbounded problem; and the supporting appendices contain incorrect or unsupported claims.

Significance. If the polynomial HGR estimator were correctly formulated and validated, it would provide a deterministic, interpretable, and differentiable alternative to neural, KDE, and randomized HGR estimators, with plausible value as a fairness regularizer. The paper also contains a correct monotonicity argument (Appendix D) and a reasonable experimental design with oracle baselines. These strengths are real, but the central algorithmic definition is currently not mathematically valid, so the significance of the contribution is not established as written.

major comments (4)

[Section 3.1, Eqs. (8)-(9)] The optimization is stated as arg max over (eα,β) of the squared residual ||eP_h^a eα − eP_k^b β / σ(eP_k^b β)||^2. This is the opposite of the least-squares minimization that is needed to maximize correlation, and it is unbounded: in the simplest case h=k=1, the constraint fixes β up to sign, while α is free, so the objective grows without bound as |α|→∞. Thus Eqs. (8)-(9) do not define a finite estimator and cannot be equivalent to Eq. (4) or to Eq. (6), which is already written as an arg min. The same error propagates to HGR-SK in Eqs. (10)-(11), where β is fixed by the variance constraint and the arg max over α is unbounded. Please correct the sign and provide a well-posed formulation with appropriate normalization or constraints.
[Appendix B, Eq. (23)] The single-level equivalence is justified by the statement 'Given that all transformations are invertible,' but polynomial maps of degree greater than one are not invertible. This invalidates the proof as written. The algebraic step from Eq. (22) to Eq. (23) does not actually require invertibility, so the equivalence may be true, but it is not established by the supplied argument. A correct derivation that does not rely on this false assumption is needed.
[Appendix C] The claimed convex Lagrangian formulation does not hold. Equation (9) is an arg max of a convex quadratic c(α,β); the equality constraint σ(eP_k^b β)^2 = 1 defines a nonconvex set, and the proposed penalty p(α,β)=|σ(eP_k^b β)^2−1| is not convex. The inequalities in Eqs. (26)-(28) are only monotonicity relations for minimizers of c+μp and do not imply convexity, existence of a global optimum, or convergence. Consequently, the statement that 'a globally optimal solution exists' and the use of a trust-region solver as a global optimizer are unsupported. For the arg-min version, a correct treatment would be needed (e.g., a generalized-eigenvalue reformulation rather than a Lagrangian convexity argument).
[Section 3.3, Differentiability paragraph] The claim that HGR-KB yields a 'valid subgradient' through Eq. (7) is not derived. The solution (eα*,β*) of a constrained optimization problem is generally nonsmooth as a function of the data, and no chain-rule, envelope-theorem, or automatic-differentiation argument is supplied for HGR-KB. The fairness experiments in Section 4.2 rely on this subgradient for back-propagation, so this is a load-bearing gap that needs a precise derivation or a reference.

minor comments (2)

[Eq. (10)] The statement that β is 'completely determined by the constraint and equal to 1/σ(b)' omits the sign; the constraint only fixes β = ±1/σ(b).
[Abstract / Introduction] The phrase 'inherent uncomputability of HGR' is imprecise. The population HGR coefficient is a well-defined supremum; it is not computable from finite samples in general, but this is a sample-complexity issue rather than an inherent uncomputability of the coefficient itself.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity; the estimator is a direct optimization and is benchmarked against external oracles; the only self-citations are motivational and non-load-bearing.

full rationale

The paper's central estimator HGR-KB is defined by an explicit optimization over polynomial coefficients (Eq. 4) and is evaluated against external ground-truth oracles (synthetic data) and independent baselines (HGR-NN, HGR-KDE, RDC); no parameter is fitted to benchmark outcomes, so there is no fitted-input-called-prediction. The reformulation in Eqs. (6)-(9) is an algebraic rewriting of the same least-squares objective; although the printed arg max in Eqs. (8)-(9) appears to have the wrong sign (a correctness concern, not a circularity), it is not an equivalence-by-construction between inputs and outputs. The monotonicity result in Appendix D follows from embedding lower-degree coefficient vectors into higher-degree spaces, i.e., enlarging the feasible set; it is a direct consequence, not a circular derivation. The only self-citations are Giuliani et al. (2023) for the known overfitting risk of sample HGR and Fioretto et al. (2021) for the Lagrangian dual training framework; both are motivational or methodological background and are not load-bearing for the derivation of HGR-KB. Thus the paper is self-contained and no circular step reduces the claimed result to its inputs.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The central estimator rests on a standard least-squares equivalence, but also on non-invertibility and on an unproven convexity claim for the resulting QCQP. These are load-bearing assumptions that are not justified by the appendix.

free parameters (2)

h, k (kernel degrees for HGR-KB) = 5 (default)
User-configurable polynomial orders; the paper notes the optimal trade-off is around h=k=3 in Figure 4 but defaults to 5.
d (single-kernel degree for HGR-SK) = 5 (default)
Degree of the one-sided polynomial regression; controls bias-variance and detection of functional dependencies.

assumptions (4)

ad hoc to paper The optimization problem in Equation (9) admits a convex Lagrangian formulation and a globally optimal solution exists.
Appendix C claims this, but the objective is concave (negative squared norm) and the variance constraint is nonconvex; no convexity proof is supplied.
domain assumption All copula transformations are invertible, used in Appendix B to equate squared-correlation maximization with MSE minimization.
Polynomial functions of degree greater than one are not invertible in general, so the sign-flipping argument in Equation (23) does not apply to the proposed polynomial kernels.
domain assumption Finite-degree polynomial expansions can approximate the optimal copula transformations well enough to estimate HGR.
This is the bias-variance trade-off the authors acknowledge; if the true transformation is outside the polynomial span (for example, y equals sin(x squared)), HGR-KB underestimates.
standard math Pearson correlation is equivalent to the least-squares slope of the standardized variables.
Appendix A derives this correctly; it is standard and underlies the whole reformulation.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Robust Non-Linear Correlations via Polynomial Regression." pith.science (2026). https://pith.science/paper/QXAA6HDG

@misc{pith2026250909380,
  author       = {Pith},
  title        = {Pith review of: Robust Non-Linear Correlations via Polynomial Regression},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QXAA6HDG}},
  note         = {Machine review of arXiv:2509.09380}
}

read the original abstract

The Hirschfeld-Gebelein-R\'enyi (HGR) correlation coefficient is an extension of Pearson's correlation that is not limited to linear correlations, with potential applications in algorithmic fairness, scientific analysis, and causal discovery. Recently, novel algorithms to estimate HGR in a differentiable manner have been proposed to facilitate its use as a loss regularizer in constrained machine learning applications. However, the inherent uncomputability of HGR requires a bias-variance trade-off, which can possibly compromise the robustness of the proposed methods, hence raising technical concerns if applied in real-world scenarios. We introduce a novel computational approach for HGR that relies on user-configurable polynomial kernels, offering greater robustness compared to previous methods and featuring a faster yet almost equally effective restriction. Our approach provides significant advantages in terms of robustness and determinism, making it a more reliable option for real-world applications. Moreover, we present a brief experimental analysis to validate the applicability of our approach within a constrained machine learning framework, showing that its computation yields an insightful subgradient that can serve as a loss regularizer.

Figures

Figures reproduced from arXiv: 2509.09380 by the authors.

**Figure 2.** Time required to compute HGR-SK using Least-Square vs. Global Optimization algorithms. following bi-level optimization definition for HGR(a, b): max f,g arg min r [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

**Figure 3.** Example of kernel computation. relationship of the yielded correlation in both directions. Moreover, it shows that the optimal trade-off for kernel degrees is likely to be found in h, k ≈ 3, since lower values would result in an underestimation of the correlation, while higher values would bring little to no improvement. It is worth mentioning that, although these parameters are data-specific, their calibration is m… view at source ↗

Figures from the paper (4 more)

**Figure 4.** Figure 4: Computation of HGR-KB(a, b; h, k) with varying h and k on three benchmark datasets. 6 [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]

**Figure 5.** Figure 5: Effects of algorithm stochasticity in three different techniques to estimate HGR. [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]

**Figure 6.** Figure 6: Execution times and correlations computed using several indicators across distinct deterministic relationships [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]

**Figure 7.** Figure 7: Kernel inspection of three HGR indicators on circular dataset. [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

14 extracted references · 4 canonical work pages

[1]

, author Jordan, M

author Bach, F. , author Jordan, M. , year 2003 . title Kernel independent component analysis . journal Journal of Machine Learning Research volume 3 , pages 1--48 . :10.1162/153244303768966085

doi:10.1162/153244303768966085 2003
[2]

, author Friedman, J.H

author Breiman, L. , author Friedman, J.H. , year 1985 . title Estimating optimal transformations for multiple regression and correlation . journal Journal of the American Statistical Association volume 80 , pages 580–598 . http://dx.doi.org/10.1080/01621459.1985.10478157, :10.1080/01621459.1985.10478157

arXiv 1985
[3]

, author Gould, N.I.M

author Conn, A.R. , author Gould, N.I.M. , author Toint, P.L. , year 2000 . title Trust Region Methods . publisher Society for Industrial and Applied Mathematics . http://dx.doi.org/10.1137/1.9780898719857, :10.1137/1.9780898719857

doi:10.1137/1.9780898719857 2000
[4]

, author Hentenryck, P.V

author Fioretto, F. , author Hentenryck, P.V. , author Mak, T.W.K. , author Tran, C. , author Baldo, F. , author Lombardi, M. , year 2021 . title Lagrangian duality for constrained deep learning , in: booktitle Machine Learning and Knowledge Discovery in Databases. Applied Data Science and Demo Track . publisher Springer International Publishing , pp. pag...

work page doi:10.1007/978-3-030-67670-4_8 2021
[5]

, author Misino, E

author Giuliani, L. , author Misino, E. , author Lombardi, M. , year 2023 . title Generalized disparate impact for configurable fairness solutions in ML , in: editor Krause, A. , editor Brunskill, E. , editor Cho, K. , editor Engelhardt, B. , editor Sabato, S. , editor Scarlett, J. (Eds.), booktitle Proceedings of the 40th International Conference on Mach...

2023
[6]

, author Lamprier, S

author Grari, V. , author Lamprier, S. , author Detyniecki, M. , year 2020 . title Fairness-aware neural rényi minimization for continuous features , in: editor Bessiere, C. (Ed.), booktitle Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, IJCAI-20 , publisher International Joint Conferences on Artificial Intellig...

work page doi:10.24963/ijcai.2020/313 2020
[7]

, author Herbrich, R

author Gretton, A. , author Herbrich, R. , author Smola, A. , author Bousquet, O. , author Sch\" o lkopf, B. , year 2005 . title Kernel methods for measuring independence . journal J. Mach. Learn. Res. volume 6 , pages 2075–2129

2005
[8]

, author Shawe-Taylor, J

author Hardoon, D.R. , author Shawe-Taylor, J. , year 2008 . title Convergence analysis of kernel canonical correlation analysis: theory and practice . journal Machine Learning volume 74 , pages 23–38 . http://dx.doi.org/10.1007/s10994-008-5085-3, :10.1007/s10994-008-5085-3

work page doi:10.1007/s10994-008-5085-3 2008

Show all 14 references

[9]

, author Hennig, P

author Lopez-Paz, D. , author Hennig, P. , author Sch\" o lkopf, B. , year 2013 . title The randomized dependence coefficient , in: editor Burges, C. , editor Bottou, L. , editor Welling, M. , editor Ghahramani, Z. , editor Weinberger, K. (Eds.), booktitle Advances in Neural I...

2013
[10]

, author Calauz \`e nes, C

author Mary, J. , author Calauz \`e nes, C. , author Karoui, N.E. , year 2019 . title Fairness-aware learning for continuous attributes and treatments , in: editor Chaudhuri, K. , editor Salakhutdinov, R. (Eds.), booktitle Proceedings of the 36th International Conference on Ma...

2019
[11]

, author Ghahramani, Z

author P\' o czos, B. , author Ghahramani, Z. , author Schneider, J. , year 2012 . title Copula-based kernel dependency measures , in: booktitle Proceedings of the 29th International Coference on International Conference on Machine Learning , publisher Omnipress , address Madi...

2012
[12]

, year 1959

author R \'e nyi, A. , year 1959 . title On measures of dependence . journal Acta Mathematica Academiae Scientiarum Hungarica volume 10 , pages 441--451

1959
[13]

, author Rizzo, M.L

author Székely, G.J. , author Rizzo, M.L. , year 2009 . title Brownian distance covariance . journal The Annals of Applied Statistics volume 3 . http://dx.doi.org/10.1214/09-AOAS312, :10.1214/09-aoas312

2009 doi
[14]

, year 1975

author Witsenhausen, H.S. , year 1975 . title On sequences of pairs of dependent random variables . journal SIAM Journal on Applied Mathematics volume 28 , pages 100–113 . http://dx.doi.org/10.1137/0128010, :10.1137/0128010

1975 doi

Pith tools

Reviewed August 4, 2026 · model on record in the stance chip above.

[1] [1]

, author Jordan, M

author Bach, F. , author Jordan, M. , year 2003 . title Kernel independent component analysis . journal Journal of Machine Learning Research volume 3 , pages 1--48 . :10.1162/153244303768966085

doi:10.1162/153244303768966085 2003

[2] [2]

, author Friedman, J.H

author Breiman, L. , author Friedman, J.H. , year 1985 . title Estimating optimal transformations for multiple regression and correlation . journal Journal of the American Statistical Association volume 80 , pages 580–598 . http://dx.doi.org/10.1080/01621459.1985.10478157, :10.1080/01621459.1985.10478157

arXiv 1985

[3] [3]

, author Gould, N.I.M

author Conn, A.R. , author Gould, N.I.M. , author Toint, P.L. , year 2000 . title Trust Region Methods . publisher Society for Industrial and Applied Mathematics . http://dx.doi.org/10.1137/1.9780898719857, :10.1137/1.9780898719857

doi:10.1137/1.9780898719857 2000

[4] [4]

, author Hentenryck, P.V

author Fioretto, F. , author Hentenryck, P.V. , author Mak, T.W.K. , author Tran, C. , author Baldo, F. , author Lombardi, M. , year 2021 . title Lagrangian duality for constrained deep learning , in: booktitle Machine Learning and Knowledge Discovery in Databases. Applied Data Science and Demo Track . publisher Springer International Publishing , pp. pag...

work page doi:10.1007/978-3-030-67670-4_8 2021

[5] [5]

, author Misino, E

author Giuliani, L. , author Misino, E. , author Lombardi, M. , year 2023 . title Generalized disparate impact for configurable fairness solutions in ML , in: editor Krause, A. , editor Brunskill, E. , editor Cho, K. , editor Engelhardt, B. , editor Sabato, S. , editor Scarlett, J. (Eds.), booktitle Proceedings of the 40th International Conference on Mach...

2023

[6] [6]

, author Lamprier, S

author Grari, V. , author Lamprier, S. , author Detyniecki, M. , year 2020 . title Fairness-aware neural rényi minimization for continuous features , in: editor Bessiere, C. (Ed.), booktitle Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, IJCAI-20 , publisher International Joint Conferences on Artificial Intellig...

work page doi:10.24963/ijcai.2020/313 2020

[7] [7]

, author Herbrich, R

author Gretton, A. , author Herbrich, R. , author Smola, A. , author Bousquet, O. , author Sch\" o lkopf, B. , year 2005 . title Kernel methods for measuring independence . journal J. Mach. Learn. Res. volume 6 , pages 2075–2129

2005

[8] [8]

, author Shawe-Taylor, J

author Hardoon, D.R. , author Shawe-Taylor, J. , year 2008 . title Convergence analysis of kernel canonical correlation analysis: theory and practice . journal Machine Learning volume 74 , pages 23–38 . http://dx.doi.org/10.1007/s10994-008-5085-3, :10.1007/s10994-008-5085-3

work page doi:10.1007/s10994-008-5085-3 2008