RECORD
Improving cosmological covariance matrices with machine learning
Cosmology and Nongalactic Astrophysics · Submitted 22 May 2022
abstract compiled · 2026-09-23 · no language-model review
The authors' main claim Finally, we show that, by using the denoised matrices, the cosmological parameters can be recovered with nearly the same accuracy as when using matrices built with a sample of 30,000 spectra in the case of the cheap simulations, and with 15,000 spectra in the case of the N-body simulations.
The compiled record
The paper's own sentences, arranged. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. Numbers in the tables are the authors', checked by code. No sentence in this record was written by a model.
Open the compiled record
The abstract, sentence by sentence
Cosmological covariance matrices are fundamental for parameter inference, since they are responsible for propagating uncertainties from the data down to the model parameters. However, when data vectors are large, in order to estimate accurate and precise matrices we need huge numbers of observations, or rather costly simulations - neither of which may be viable. In this work we propose a machine learning approach to alleviate this problem in the context of the matrices used in the study of large-scale structure. With only a small amount of data (matrices built with samples of 50-200 halo power spectra) we are able to provide significantly improved matrices, which are almost indistinguishable from the ones built from much larger samples (thousands of spectra). In order to perform this task we trained convolutional neural networks to denoise the matrices, using in the training process a data set made up entirely of spectra extracted from simple, inexpensive halo simulations (mocks). We then show that the method not only removes the noise in the matrices of the cheap simulation, but it is also able to successfully denoise the matrices of halo power spectra from N-body simulations. We compare the denoised to the other matrices using several metrics, and in all of them they score better, without any signs of spurious artifacts. With the help of the Wishart distribution we derive an analytical extrapolation for the effective sample augmentation allowed by the denoiser. Finally, we show that, by using the denoised matrices, the cosmological parameters can be recovered with nearly the same accuracy as when using matrices built with a sample of 30,000 spectra in the case of the cheap simulations, and with 15,000 spectra in the case of the N-body simulations. Of particular interest is the bias in the Hubble parameter $H_0$, which was significantly reduced after applying the denoiser.
background contribution method result interpretation main claim
| # | sentence | role | the authors' own work | states the main conclusion | strength |
|---|---|---|---|---|---|
| s0 | Cosmological covariance matrices are fundamental for parameter inference, since they are responsible for propagating uncertainties from the data down to the model parameters. | background 1.00 | 0.05 | 0.02 | asserted 1.00 |
| s1 | However, when data vectors are large, in order to estimate accurate and precise matrices we need huge numbers of observations, or rather costly simulations - neither of which may be viable. | background 0.69 | 0.08 | 0.02 | asserted 0.69 |
| s2 | In this work we propose a machine learning approach to alleviate this problem in the context of the matrices used in the study of large-scale structure. | contribution 1.00 | 0.82 | 0.07 | asserted 0.87 |
| s3 | With only a small amount of data (matrices built with samples of 50-200 halo power spectra) we are able to provide significantly improved matrices, which are almost indistinguishable from the ones built from much larger samples (thousands of spectra). | result 0.96 | 0.96 | 0.60 | asserted 1.00 |
| s4 | In order to perform this task we trained convolutional neural networks to denoise the matrices, using in the training process a data set made up entirely of spectra extracted from simple, inexpensive halo simulations (mocks). | method 1.00 | 0.96 | 0.05 | asserted 0.99 |
| s5 | We then show that the method not only removes the noise in the matrices of the cheap simulation, but it is also able to successfully denoise the matrices of halo power spectra from N-body simulations. | result 0.97 | 0.96 | 0.12 | asserted 1.00 |
| s6 | We compare the denoised to the other matrices using several metrics, and in all of them they score better, without any signs of spurious artifacts. | result 1.00 | 0.96 | 0.12 | asserted 1.00 |
| s7 | With the help of the Wishart distribution we derive an analytical extrapolation for the effective sample augmentation allowed by the denoiser. | method 0.52 | 0.95 | 0.05 | asserted 1.00 |
| s8 | Finally, we show that, by using the denoised matrices, the cosmological parameters can be recovered with nearly the same accuracy as when using matrices built with a sample of 30,000 spectra in the case of the cheap simulations, and with 15,000 spectra in the case of the N-body simulations. | result 1.00 | 0.97 | 0.51 | asserted 1.00 |
| s9 | Of particular interest is the bias in the Hubble parameter $H_0$, which was significantly reduced after applying the denoiser. | result 0.97 | 0.94 | 0.11 | asserted 1.00 |
What was examined, and what was not
- arXiv listing abstract v2
- full text
- figures and tables
- references and citing sentences
- relations to other papers
- P abstract
- S sections
- T body sentences
- C citations
- R cross-paper relations
The paper
Original abstract
Cosmological covariance matrices are fundamental for parameter inference, since they are responsible for propagating uncertainties from the data down to the model parameters. However, when data vectors are large, in order to estimate accurate and precise matrices we need huge numbers of observations, or rather costly simulations - neither of which may be viable. In this work we propose a machine learning approach to alleviate this problem in the context of the matrices used in the study of large-scale structure. With only a small amount of data (matrices built with samples of 50-200 halo power spectra) we are able to provide significantly improved matrices, which are almost indistinguishable from the ones built from much larger samples (thousands of spectra). In order to perform this task we trained convolutional neural networks to denoise the matrices, using in the training process a data set made up entirely of spectra extracted from simple, inexpensive halo simulations (mocks). We then show that the method not only removes the noise in the matrices of the cheap simulation, but it is also able to successfully denoise the matrices of halo power spectra from N-body simulations. We compare the denoised to the other matrices using several metrics, and in all of them they score better, without any signs of spurious artifacts. With the help of the Wishart distribution we derive an analytical extrapolation for the effective sample augmentation allowed by the denoiser. Finally, we show that, by using the denoised matrices, the cosmological parameters can be recovered with nearly the same accuracy as when using matrices built with a sample of 30,000 spectra in the case of the cheap simulations, and with 15,000 spectra in the case of the N-body simulations. Of particular interest is the bias in the Hubble parameter $H_0$, which was significantly reduced after applying the denoiser.
Human review
References
How the literature uses it
5 arXiv papers cite this in 5 sentences
Papers citing it · 2 sentences without a confident label
Cites it as background 3 papers
-
In this respect, over the last years, several works addressed the problem of a poor numerical estimate of the precision matrix [ref] , while various strategies to reduce the number of required realizations have been explored [ref] .
Fitting covariance matrix models to simulations · Introduction · 0.85
-
Among others, these techniques have been used to learn the spatial distribution of dark matter tracers from a large number of detailed $N$-body simulations [ref] , generate corrections to the displacement field in Lagrangian perturbation theory [ref] , increase the mass resolution of fast and computationally cheap simulations [ref] , learn the galaxy–dark matter connection from hydro-simulations [ref] , and to provide a platform to obtain covariance matrices from fast and/or inaccurate sets of mocks [ref] .
DESI Mock Challenge: Halo and galaxy catalogs with the bias assignment method · Introduction · 0.93
-
There have also been recent developments in using neural networks to denoise the sample covariance matrix [ref] , though training such models may be computationally prohibitive for very large $n$.
A multi-eigenbasis approach to covariance matrix denoising for cosmological inference · Introduction · 0.86
What it cites · 47 sentences without a confident label
Uses the method or tool of 7 papers
-
ExSHalos (Excursion Set Halos) constitutes a new, simple, fast, and parameter-free method to generate dark matter halo catalogues [ref] .
Excursion Set Halos -- ExSHalos: A New Parameter Free Method for Fast Generation of Halo Catalogues · ExSHalos · 0.88
-
The simulations employed dark matter-only particles using the TreePM code GADGET-III – the third generation of the well known GADGET-II algorithm [ref] .
The cosmological simulation code GADGET-2 · Quijote · 0.97
-
For the ExSHalos mocks we used a linear power spectrum from the Code for Anisotropies in the Microwave Background ( https://camb.info/ CAMB) [ref] .
The linear power spectrum of observed source number counts · ExSHalos · 0.85
Uses data or values from 1 paper
-
The suite has simulations for a range of cosmological models, but the main (standard) fiducial cosmology follows the Planck best-fit model [ref] : $Ω_m = 0.3175$, $Ω_b = 0.049$, $h = 0.6711$, $n_s = 0.9624$, $σ_8 = 0.834$, $M_ν = 0.0 eV$ and $ω= - 1$.
Planck 2018 results. VI. Cosmological parameters · Quijote · 0.86
Compares with 5 papers
-
We have shown that the proposed method can be matched in terms of an extrapolation according to the Wishart distribution [ref] , by effectively augmenting the size of the sample that underlies the covariance matrix.
Putting the Precision in Precision Cosmology: How accurate should your data covariance matrix be? · Discussion and Conclusions · 0.92
-
We then compared the resulting halo mass functions from the catalogues with the fits by Tinker et al. [ref] (using $ = 200$ to compare with Quijote halos, found with link parameter $b = 0.2$) and Bhattacharya et al. [ref] , which were computed with the help of the https://bdiemer.bitbucket.io/colossus/lss_mass_function.htmlColossus library [ref] .
Toward a halo mass function for precision cosmology: the limits of universality · The match between the catalogues · 0.91
-
We then compared the resulting halo mass functions from the catalogues with the fits by Tinker et al. [ref] (using $ = 200$ to compare with Quijote halos, found with link parameter $b = 0.2$) and Bhattacharya et al. [ref] , which were computed with the help of the https://bdiemer.bitbucket.io/colossus/lss_mass_function.htmlColossus library [ref] .
Mass Function Predictions Beyond LCDM · The match between the catalogues · 0.91
Cites as support 4 papers
-
In this way, we expected to provide matrices that can even circumvent the well known errors (up to 10%) in the parameter inference, that occur when using matrices that were derived only from mocks or other approximated methods [ref] .
Comparing approximate methods for mock catalogues and covariance matrices III: Bispectrum · Discussion and Conclusions · 0.96
-
In this way, we expected to provide matrices that can even circumvent the well known errors (up to 10%) in the parameter inference, that occur when using matrices that were derived only from mocks or other approximated methods [ref] .
Comparing approximate methods for mock catalogues and covariance matrices I: correlation function · Discussion and Conclusions · 0.96
-
In this way, we expected to provide matrices that can even circumvent the well known errors (up to 10%) in the parameter inference, that occur when using matrices that were derived only from mocks or other approximated methods [ref] .
Comparing approximate methods for mock catalogues and covariance matrices II: Power spectrum multipoles · Discussion and Conclusions · 0.96
Cites as background 31 papers
-
Several efforts have already been made with the goal of obtaining precise covariance matrices using smaller samples – for a review see Ref. [ref] .
Putting the Precision in Precision Cosmology: How accurate should your data covariance matrix be? · Introduction · 0.96
-
In Refs. [ref] the authors try to emulate the full non-linear evolution of N-body simulations by inputting only approximate simulations of these.
NECOLA: Towards a Universal Field-level Cosmological Emulator · Introduction · 0.93
-
In Refs. [ref] the authors try to emulate the full non-linear evolution of N-body simulations by inputting only approximate simulations of these.
Learning to Predict the Cosmological Structure Formation · Introduction · 0.93
Discussion
Know something about this paper? Leave a comment, or highlight any passage above to comment on it.