Denoising clustering covariance matrices with Rotational Invariant Estimators

Alfonso Veropalumbo; Antonio Farina; Claudio Guida; Massimo Guidi

arxiv: 2604.13851 · v1 · submitted 2026-04-15 · 🌌 astro-ph.CO

Denoising clustering covariance matrices with Rotational Invariant Estimators

Antonio Farina , Massimo Guidi , Alfonso Veropalumbo , Claudio Guida This is my paper

Pith reviewed 2026-05-10 12:35 UTC · model grok-4.3

classification 🌌 astro-ph.CO

keywords covariance matrix estimationrotational invariant estimatorgalaxy clusteringcosmological inferencemock catalogstwo-point correlation functionpower spectrumprecision matrix

0 comments

The pith

The Rotational Invariant Estimator stabilizes best-fit recovery in galaxy clustering analyses even with few mocks.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper applies the Rotational Invariant Estimator (RIE) to covariance matrices derived from limited mock catalogs for galaxy two-point clustering statistics. It benchmarks RIE against the standard sample covariance and the NERCOME shrinkage estimator on synthetic data for both the correlation function and power spectrum. The results show that RIE reduces the stochastic biases and scatter in cosmological parameter fits that arise when the number of mocks approaches the data-vector dimension, with especially strong performance in Fourier space.

Core claim

Among the estimators tested, RIE emerges as the most effective at stabilizing best-fit recovery, particularly in Fourier space, where it closely reproduces the reference posteriors even when the number of mocks barely exceeds the data-vector dimension.

What carries the argument

The Rotational Invariant Estimator (RIE), which produces a denoised covariance matrix by exploiting rotational invariance properties of the estimator.

Load-bearing premise

The controlled synthetic data sets with analytically known covariance matrices accurately represent the statistical properties and noise structure encountered in real galaxy clustering observations.

What would settle it

Repeating the full inference pipeline on the same synthetic data but with a much larger number of mocks to serve as ground truth, then checking whether RIE still yields best-fit values and posterior volumes that match the high-mock reference within expected statistical fluctuations.

Figures

Figures reproduced from arXiv: 2604.13851 by Alfonso Veropalumbo, Antonio Farina, Claudio Guida, Massimo Guidi.

**Figure 1.** Figure 1: Left panels: FoB of the 2PCF of the synthetic dataset as a function of q for the sample (red), NERCOME (gold) and RIE (blue) estimators. Dots and error bars are, respectively, mean and rms scatter of the FoB as obtained from 100 different realizations of the covariance matrix. Each row correspond to a different datavector length D as reported in the panel titles. Right panels: Same plots but for the FoM… view at source ↗

**Figure 2.** Figure 2: Normalised distributions of the best-fit parameter estimates for h (left column) and ωc (right column) derived from the 2PCF for different data vector sizes D. Results are shown for NERCOME (gold), RIE (blue), and sample covariance (red). The vertical black dashed lines indicate the true parameter values inferred from the exact covariance, while the cyan dashed lines show the distribution of best-fit valu… view at source ↗

**Figure 3.** Figure 3: Posterior distributions for the model parameters obtained from the 2PCF analysis for D = 270 using different covariance matrix estimators. For each method, we show the realization whose FoM and FoB are closest to the median values of their respective distributions. Results are shown for two values of q (q = 1.2 and q = 16), comparing sample (red), NERCOME (gold), RIE (blue), and the reference case (grey). … view at source ↗

**Figure 4.** Figure 4: Left panel: FoB of the power spectrum of the synthetic dataset as a function of the number of mocks used to compute the covariance matrix for the sample (red), NERCOME (gold) and RI (blue) estimators. Dots and errorbars are, respectively, mean and rms scatter of the FoB as obtained from 100 different realizations of the covariance matrix. Each row correspond to a different datavector length D as reported i… view at source ↗

**Figure 5.** Figure 5: Normalised distributions of the best-fit parameter estimates for h (left column) and ωc (right column) derived from the power spectrum for different data vector sizes D. Results are shown for NERCOME (gold), RIE (blue), and sample covariance (red). The vertical black dashed lines indicate the true parameter values inferred from the exact covariance, while the cyan dashed lines show the distribution of be… view at source ↗

**Figure 6.** Figure 6: Posterior distributions for the model parameters obtained from the power spectrum analysis for D = 270 using different covariance matrix estimators. For each method, we show the realization whose FoM and FoB are closest to the median values of their respective distributions. Results are shown for two values of q (q = 1.2 and q = 16), comparing sample (red), NERCOME (gold), RIE (blue), and the reference cas… view at source ↗

read the original abstract

Cosmological parameter inference from galaxy clustering relies critically on accurate estimates of the covariance and precision matrices. These are often obtained from a limited number of mock catalogs, introducing noise and bias in the precision matrix when the data-vector dimension becomes comparable to the number of available realizations. We present the first application of the Rotational Invariant Estimator (RIE) to the large-scale clustering of galaxies, benchmarking it against the standard sample covariance and the non-linear shrinkage estimator NERCOME for both the two-point correlation function (2PCF) and power spectrum. Using controlled synthetic data sets with analytically known covariance matrices, we estimate the covariance with all three methods across a range of mock-to-dimension ratios $q = N/D$ and data-vector sizes $D$. We then perform Bayesian inference with an EFT-based model and quantify each estimator through the Figure of Bias (FoB) and Figure of Merit (FoM). After correction for finite-$N$ effects, the sample covariance recovers unbiased average uncertainty volumes but suffers from growing best-fit scatter and bias at small $q$ due to the Dodelson--Schneider effect. Both NERCOME and RIE substantially reduce these stochastic shifts; however, the uncertainties they assign are probe-dependent. In configuration space, both estimators can yield overly tight constraints, with a bias that grows with $D$. In Fourier space, RIE delivers markedly improved best-fit stability with only mild FoM bias, whereas NERCOME tends to overestimate the constraining power. Among the estimators tested, RIE emerges as the most effective at stabilizing best-fit recovery, particularly in Fourier space, where it closely reproduces the reference posteriors even when the number of mocks barely exceeds the data-vector dimension.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

RIE gives more stable best-fit recovery than sample covariance or NERCOME for clustering covariances when mocks are limited, at least on these synthetic tests.

read the letter

The core result is that the Rotational Invariant Estimator reduces scatter and bias in recovered parameters compared with the usual sample covariance and NERCOME, particularly for power-spectrum data vectors when the number of mocks is only a bit larger than the dimension. They are the first to test RIE on galaxy clustering covariances for both 2PCF and power spectrum. The evaluation uses synthetic data with analytically known truth, applies finite-N corrections to the sample estimator, and measures performance with Figure of Bias and Figure of Merit after running EFT-based inference. This controlled setup lets them show that RIE keeps posteriors close to the reference even at low q, while the other methods degrade more noticeably in Fourier space. The comparisons are systematic across several q values and data-vector sizes, which is the right way to benchmark an estimator for this use case. The work is grounded because the reference posteriors come from the known covariance rather than from quantities defined inside the paper itself. The main limitation is that everything stays on synthetic data whose noise properties may not fully match real survey data, including geometry effects and non-Gaussian contributions. In configuration space the shrinkage estimators can produce overly tight constraints whose bias grows with dimension. Implementation choices for the RIE are not spelled out in enough detail to judge sensitivity to those decisions. This paper is for people who build or use covariance matrices for large-scale structure parameter inference and need a practical way to handle small mock numbers. A reader working on survey analyses will find the quantitative benchmarks useful. It deserves peer review because the tests are clean, the improvement is measured directly, and the scope is a narrow but important step in the pipeline.

Referee Report

0 major / 4 minor

Summary. The manuscript presents the first application of the Rotational Invariant Estimator (RIE) to covariance matrix estimation for galaxy clustering, benchmarking it against the sample covariance estimator (with finite-N corrections) and the NERCOME shrinkage estimator. Using synthetic datasets with analytically known true covariances, the authors compare performance for both the two-point correlation function and power spectrum across a range of mock-to-dimension ratios q = N/D. They quantify results via Figure of Bias (FoB) and Figure of Merit (FoM) after Bayesian inference with an EFT-based model, concluding that RIE provides the best stabilization of best-fit parameter recovery (especially in Fourier space) while mitigating Dodelson-Schneider effects, though uncertainty calibration is probe-dependent.

Significance. If the benchmark results hold, the work offers a practical advance for cosmological parameter inference when the number of mocks is comparable to or only modestly exceeds the data-vector dimension, a common limitation in analyses of surveys such as DESI or Euclid. Credit is due for the controlled synthetic setup with analytically known truth, which enables direct, independent validation of FoB/FoM metrics rather than circular self-consistency tests, and for the explicit finite-N corrections applied to the sample estimator.

minor comments (4)

The abstract and methods description would benefit from an explicit early definition of the ratio q = N/D and a brief statement of how the finite-N correction is implemented for the sample covariance (e.g., which formula or reference is used).
Implementation details for the RIE (e.g., any regularization parameters, choice of rotationally invariant shrinkage form, or numerical stability checks) are not fully specified; adding a short paragraph or appendix would improve reproducibility.
The text should clarify whether the reported FoM and FoB values are averaged over multiple independent realizations of the synthetic data or derived from a single run, and how error bars on these metrics are obtained.
Figure captions and axis labels could be expanded to indicate the exact data-vector dimension D and the number of mocks N for each panel, rather than relying solely on the q values.

Simulated Author's Rebuttal

0 responses · 0 unresolved

We thank the referee for their careful reading of the manuscript, positive summary of our results, and recommendation for minor revision. The controlled synthetic setup with known truth and the focus on practical performance at modest q = N/D are indeed central to the work. We address the major comments below.

Circularity Check

0 steps flagged

No significant circularity

full rationale

The paper conducts an empirical benchmarking study on synthetic datasets where the true covariance matrix is known analytically by construction. Reference posteriors are obtained directly from this known truth using an EFT-based model, and the FoB/FoM metrics compare estimator performance against these external references. No derivation, prediction, or central claim reduces by the paper's equations to a fitted input or self-defined quantity. Any references to prior RIE work or finite-N corrections (e.g., Dodelson-Schneider) are external to the benchmarking results and do not form a load-bearing self-citation chain that would force the reported outcomes.

Axiom & Free-Parameter Ledger

0 free parameters · 2 axioms · 0 invented entities

Based on the abstract alone, the central claim rests on standard domain assumptions about synthetic data fidelity and the suitability of the chosen inference model and metrics; no new entities or fitted parameters are described.

axioms (2)

domain assumption Synthetic data sets with analytically known covariance matrices faithfully reproduce the statistical properties of real galaxy clustering observations
Used to benchmark all estimators and quantify FoB/FoM
domain assumption The EFT-based model provides an adequate description for the Bayesian inference step used to evaluate estimator performance
Invoked when computing Figure of Bias and Figure of Merit

pith-pipeline@v0.9.0 · 5615 in / 1390 out tokens · 40168 ms · 2026-05-10T12:35:24.611348+00:00 · methodology

discussion (0)

Reference graph

Works this paper leans on

2 extracted references · 2 canonical work pages

[1]

D’Amico, Y

Abadir, K. M., Distaso, W., & Žikeš, F. 2014, Journal of Econometrics, 181, 165 Article number, page 8 of 11 A. Farina et al.: Denoising cosmological covariance matrices with Rotational Invariant estimators Fig. 6.Posterior distributions for the model parameters obtained from the power spectrum analysis forD=270 using different covariance matrix estimator...

work page arXiv 2014
[2]

as a function of the cosmological parametersh andω c. The predicted basis functions are then contracted with the EFT bias and counterterm coefficients at each step of the MCMC chain, which allows a single set of trained networks to serve any combination of nuisance parameters without retrain- ing. Input preprocessing.The two raw cosmological inputs (h, ω ...

work page 2020

[1] [1]

D’Amico, Y

Abadir, K. M., Distaso, W., & Žikeš, F. 2014, Journal of Econometrics, 181, 165 Article number, page 8 of 11 A. Farina et al.: Denoising cosmological covariance matrices with Rotational Invariant estimators Fig. 6.Posterior distributions for the model parameters obtained from the power spectrum analysis forD=270 using different covariance matrix estimator...

work page arXiv 2014

[2] [2]

as a function of the cosmological parametersh andω c. The predicted basis functions are then contracted with the EFT bias and counterterm coefficients at each step of the MCMC chain, which allows a single set of trained networks to serve any combination of nuisance parameters without retrain- ing. Input preprocessing.The two raw cosmological inputs (h, ω ...

work page 2020