Full Nonlinear Velocity Reconstruction With Transformer and Ensemble Tree Machine Learning
Pith reviewed 2026-06-28 04:39 UTC · model grok-4.3
The pith
Machine learning models trained on simulation residuals recover nonlinear galaxy velocities more accurately than linear theory across wider scales.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
Training both a gradient boosting decision tree and a Transformer to predict the residual between the actual velocity and the linear theory estimate for line-of-sight and transverse components allows the models to capture nonlinear contributions and outperform linear theory in velocity reconstruction on AbacusSummit mocks for DESI surveys.
What carries the argument
Residual prediction between true velocity and linear theory estimate, using multi-scale features fed to Transformer or gradient boosting tree models.
If this is right
- More accurate recovery of the velocity power spectrum than linear theory alone.
- Higher cross-correlation with true velocities maintained across a wider range of spatial scales.
- Improved estimates of cluster pairwise velocity correlations and stacked cluster density profiles for kSZ analyses.
Where Pith is reading between the lines
- The same residual-learning setup could be retrained on mocks that include different galaxy-halo prescriptions to test robustness before application to real catalogs.
- Velocity fields reconstructed this way might tighten constraints on modified gravity or dark energy when combined with existing density-field analyses.
Load-bearing premise
The AbacusSummit mocks accurately capture the nonlinear velocity field and survey systematics that will appear in real DESI and LSST data.
What would settle it
Training the models on one set of mocks and testing on an independent simulation suite or real DESI data yields no improvement over linear theory in power spectrum recovery or cross-correlation.
Figures
read the original abstract
Accurate reconstruction of peculiar velocities from galaxy positions is important for probing the motion and evolution of large scale structure. They are sensitive to the cosmological effects of gravity and dark sector matter, and complement other velocity inference methods such as the kinematic Sunyaev-Zel'dovich (kSZ) imprinted in the CMB. We show that machine learning methods improve velocity reconstruction by capturing nonlinear contributions. Specifically, we train both a gradient boosting decision tree (GBDT) and a Transformer using multi-scale features to predict the residual between the actual velocity and the estimate from linear theory for both the line-of-sight and transverse components. We evaluate our approach in both periodic box and, more realistic, lightcone settings using mock galaxy catalogs from the \textsc{AbacusSummit} simulations tailored to DESI spectroscopic surveys of luminous red galaxies (LRGs) and emission line galaxies (ELGs). We also assess the impact of redshift uncertainties such as those in Rubin LSST photometry. Both models significantly outperform the linear theory with the Transformer achieving the best performance. They more accurately recover the velocity power spectrum and maintain a higher cross-correlation with the true velocities across a wider range of spatial scales. Finally, we demonstrate two applications relevant to kSZ analyses: estimating cluster pairwise velocity correlations and stacked cluster density profiles. This machine learning framework for nonlinear velocity reconstruction opens up powerful new applications of survey data from DESI, Rubin LSST, Euclid and the Roman Space Telescope.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper claims that gradient boosting decision trees (GBDT) and Transformer models, trained on multi-scale features extracted from AbacusSummit mocks tailored to DESI LRG/ELG selections, can predict nonlinear velocity residuals relative to linear theory. This yields improved recovery of the velocity power spectrum and higher cross-correlation coefficients with true velocities over a wider k-range in both periodic boxes and lightcones, plus a redshift-uncertainty test for LSST; two kSZ-related applications are also shown.
Significance. If the performance gains hold under broader validation, the approach could meaningfully extend the usable k-range for peculiar-velocity analyses in DESI, LSST, Euclid and Roman data. The work supplies a concrete, simulation-grounded demonstration that nonlinear ML corrections improve upon linear theory for both line-of-sight and transverse components, which is a useful incremental step for kSZ and velocity-field cosmology.
major comments (2)
- [Abstract] Abstract and evaluation sections: the reported outperformance on power-spectrum and cross-correlation metrics is stated without quantitative error bars, without ablation of the multi-scale feature set, and without explicit documentation of training/validation splits or hyperparameter selection procedures. These omissions make it impossible to assess whether the claimed gains are statistically robust or sensitive to modeling choices.
- [Evaluation on mocks] Evaluation on mocks and applications sections: no cross-simulation validation (e.g., on a second independent simulation suite with different baryonic physics or assembly bias) is presented. Because the central motivation is application to real DESI/LSST data, the absence of any test that the reported gains survive changes in the galaxy-halo connection or unmodeled systematics is load-bearing for the transfer claim.
minor comments (2)
- Notation for the residual velocity components and the precise definition of the multi-scale features should be consolidated into a single table or equation block for clarity.
- Figure captions for the power-spectrum and cross-correlation plots should include the exact k-range over which the improvement is quantified and the number of mocks used for the error estimation.
Simulated Author's Rebuttal
We thank the referee for their thoughtful comments, which have helped us identify areas to improve the clarity and robustness of our results. We address each major comment below.
read point-by-point responses
-
Referee: [Abstract] Abstract and evaluation sections: the reported outperformance on power-spectrum and cross-correlation metrics is stated without quantitative error bars, without ablation of the multi-scale feature set, and without explicit documentation of training/validation splits or hyperparameter selection procedures. These omissions make it impossible to assess whether the claimed gains are statistically robust or sensitive to modeling choices.
Authors: We agree with this assessment. The revised manuscript will include quantitative error bars on the power spectrum and cross-correlation metrics. We will also add an ablation study to evaluate the contribution of the multi-scale feature set and provide detailed documentation of the training/validation splits and hyperparameter optimization procedures in the methods section. revision: yes
-
Referee: [Evaluation on mocks] Evaluation on mocks and applications sections: no cross-simulation validation (e.g., on a second independent simulation suite with different baryonic physics or assembly bias) is presented. Because the central motivation is application to real DESI/LSST data, the absence of any test that the reported gains survive changes in the galaxy-halo connection or unmodeled systematics is load-bearing for the transfer claim.
Authors: We recognize that cross-simulation validation would provide stronger evidence for the applicability to real data. Our work utilizes the AbacusSummit simulations, and conducting validation on additional independent suites is beyond the scope of the current study due to computational constraints. In the revision, we will add a discussion of this limitation in the conclusions and suggest it as an important direction for future research. revision: partial
Circularity Check
No significant circularity; performance measured against external simulation truth
full rationale
The paper trains GBDT and Transformer models on multi-scale features from AbacusSummit mocks to predict velocity residuals relative to linear theory, then reports power-spectrum recovery and cross-correlation on held-out test mocks (both periodic boxes and lightcones) plus a redshift-uncertainty test. These metrics are computed against the simulation's independent ground-truth velocities, not by construction from the fitted model itself. No self-citation chain, ansatz smuggling, or renaming of known results forms the load-bearing step for the central empirical claims. The derivation is therefore self-contained against the stated simulation benchmarks.
Axiom & Free-Parameter Ledger
Reference graph
Works this paper leans on
-
[1]
Pairwise Velocity Statistic The pairwise velocity is a useful summary statistic of the cosmic velocity field, characterizing the average relative in- fall of tracer pairs as a function of their comoving separation. Following [73], the pairwise velocity estimator in a separation bin centered atris, V(r)= P i<j v∥,i −v ∥,j ci j P i<j c 2 i j .(15) The sum i...
-
[2]
displacement-only
Stacked kSZ profile We also consider the application of the reconstructed ve- locities to estimate a kSZ-derived halo electron density pro- file, a useful probe of the spatial distribution of ionized gas in galaxy clusters. This is obtained by stacking the kSZ temper- atures centered on cluster halos after weighting by their LOS velocities. In our analysi...
-
[3]
Simulated kSZ Map To assess how velocity reconstruction errors propagate into a practical kSZ analysis, we construct a mock kSZ temper- ature map from the halo lightcone catalog, following [80]. In the non-relativistic limit, the kSZ temperature fluctuation along the line of sight ˆncan be written as ∆TkS Z(ˆn) T0 =− σT c Z ne vlos dl,(A1) whereσ T is the...
-
[4]
Pairwise kSZ Momentum and Optical Depth Fitting The kSZ pairwise momentum is given by PkS Z(r,z i)=− P i j(∆ ¯TAP,i −∆ ¯TAP,j)ci j P i j c2 i j ,(A11) where the sum is over all pairs of tracers with separations in a bin centered onr, and in a redshift bin centered onz.∆ ¯TAP,i is the kSZ temperature obtained from simulated maps by using aperture photometr...
-
[5]
Kaiser, Mon
N. Kaiser, Mon. Not. Roy. Astron. Soc.227, 1 (1987)
1987
-
[6]
M. A. Strauss and J. A. Willick, Phys. Rep.261, 271 (1995), arXiv:astro-ph/9502079 [astro-ph]
Pith/arXiv arXiv 1995
-
[7]
C. Howlett, K. Said, J. R. Lucey, M. Colless, F. Qin, Y . Lai, R. B. Tully, and T. M. Davis, Mon. Not. Roy. Astron. Soc.515, 953 (2022), arXiv:2201.03112 [astro-ph.CO]
arXiv 2022
-
[8]
W. J. Percival and M. White, Mon. Not. Roy. Astron. Soc.393, 297 (2009), arXiv:0808.0003 [astro-ph]
Pith/arXiv arXiv 2009
-
[9]
M. J. Hudson and S. J. Turnbull, Astrophys. J. Lett.751, L30 (2013), arXiv:1203.4814 [astro-ph.CO]
Pith/arXiv arXiv 2013
-
[10]
K. Said, M. Colless, C. Magoulas, J. R. Lucey, and M. J. Hudson, Mon. Not. Roy. Astron. Soc.497, 1275 (2020), arXiv:2007.04993 [astro-ph.CO]
arXiv 2020
-
[11]
R. J. Turner, (2024), arXiv:2411.19484 [astro-ph.CO]
arXiv 2024
-
[12]
R. A. Sunyaev and Y . B. Zeldovich, Mon. Not. Roy. Astron. Soc.190, 413 (1980)
1980
-
[13]
N. Hand et al., Phys. Rev. Lett.109, 041101 (2012), arXiv:1203.4219 [astro-ph.CO]
Pith/arXiv arXiv 2012
-
[14]
E. Schaan et al. (Atacama Cosmology Telescope), Phys. Rev. D 103, 063513 (2021), arXiv:2009.05557 [astro-ph.CO]
arXiv 2021
- [15]
-
[16]
Y . Gong et al., Phys. Rev. D113, 063538 (2026), arXiv:2511.23417 [astro-ph.CO]
arXiv 2026
-
[17]
C. M. Springob, C. Magoulas, M. Colless, J. Mould, P. Er- dogdu, D. H. Jones, J. R. Lucey, L. Campbell, and C. J. Fluke, Mon. Not. Roy. Astron. Soc.445, 2677 (2014), arXiv:1409.6161 [astro-ph.CO]
Pith/arXiv arXiv 2014
-
[18]
J. S. Bullock and M. Boylan-Kolchin, Ann. Rev. Astron. Astro- phys.55, 343 (2017), arXiv:1707.04256 [astro-ph.CO]
arXiv 2017
-
[19]
N. Battaglia, S. Ferraro, E. Schaan, and D. Spergel, JCAP11, 040, arXiv:1705.05881 [astro-ph.CO]
-
[20]
D. J. Eisenstein, H.-j. Seo, and M. J. White, Astrophys. J.664, 660 (2007), arXiv:astro-ph/0604361
Pith/arXiv arXiv 2007
-
[21]
D. J. Eisenstein, H.-j. Seo, E. Sirko, and D. Spergel, Astrophys. J.664, 675 (2007), arXiv:astro-ph/0604362
Pith/arXiv arXiv 2007
-
[22]
N. Padmanabhan, X. Xu, D. J. Eisenstein, R. Scalzo, A. J. Cuesta, K. T. Mehta, and E. Kazin, Mon. Not. Roy. Astron. Soc. 427, 2132 (2012), arXiv:1202.0090 [astro-ph.CO]
Pith/arXiv arXiv 2012
-
[23]
X. Xu, A. J. Cuesta, N. Padmanabhan, D. J. Eisenstein, and C. K. McBride, Mon. Not. Roy. Astron. Soc.431, 2834 (2013), arXiv:1206.6732 [astro-ph.CO]
Pith/arXiv arXiv 2013
-
[24]
L. Anderson et al. (BOSS), Mon. Not. Roy. Astron. Soc.439, 83 (2014), arXiv:1303.4666 [astro-ph.CO]
Pith/arXiv arXiv 2014
-
[25]
H. Gil-Marin et al. (eBOSS), Mon. Not. Roy. Astron. Soc.498, 2492 (2020), arXiv:2007.08994 [astro-ph.CO]
arXiv 2020
-
[26]
K. C. Chan, G. Lu, and X. Wang, Mon. Not. Roy. Astron. Soc. 529, 1667 (2024), arXiv:2311.12611 [astro-ph.CO]
arXiv 2024
-
[27]
M. White, Mon. Not. Roy. Astron. Soc.450, 3822 (2015), arXiv:1504.03677 [astro-ph.CO]
Pith/arXiv arXiv 2015
-
[28]
S.-F. Chen, Z. Vlah, and M. White, JCAP09, 017, arXiv:1907.00043 [astro-ph.CO]
arXiv 1907
-
[29]
E. Schaan et al. (ACTPol), Phys. Rev. D93, 082002 (2016), arXiv:1510.06442 [astro-ph.CO]
Pith/arXiv arXiv 2016
-
[30]
P. Zhang, J. Pan, and Y . Zheng, Physical Review D87, 10.1103/physrevd.87.063526 (2013)
-
[31]
C. Blake and R. J. Turner 10.33232/001c.124509 (2024), arXiv:2408.05660 [astro-ph.CO]
-
[32]
Y . Lai et al. 10.5281/zenodo.17602818 (2025), arXiv:2512.03229 [astro-ph.CO]
work page internal anchor Pith review Pith/arXiv arXiv doi:10.5281/zenodo.17602818 2025
-
[33]
A. Aghamousa et al. (DESI), (2016), arXiv:1611.00036 [astro- ph.IM]
Pith/arXiv arXiv 2016
-
[34]
LSST Science Collaboration, LSST Science Book, version 2.0 (2009), arXiv:0912.0201 [astro-ph.IM]
Pith/arXiv arXiv 2009
-
[35]
Y . Mellier et al. (Euclid), Astron. Astrophys.697, A1 (2025), arXiv:2405.13491 [astro-ph.CO]
arXiv 2025
-
[36]
Spergel et al., (2015), arXiv:1503.03757 [astro-ph.IM]
D. Spergel et al., (2015), arXiv:1503.03757 [astro-ph.IM]
Pith/arXiv arXiv 2015
-
[37]
B. R. Guachalla, E. Schaan, B. Hadzhiyska, and S. Ferraro, Phys. Rev. D109, 103533 (2024), arXiv:2312.12435 [astro- ph.CO]
arXiv 2024
-
[38]
B. Hadzhiyska, S. Ferraro, B. R. Guachalla, and E. Schaan, Phys. Rev. D109, 103534 (2024), arXiv:2312.12434 [astro- ph.CO]
arXiv 2024
-
[39]
Y . B. Zel’dovich, Astron. Astrophys.5, 84 (1970)
1970
-
[40]
X. Xu, N. Padmanabhan, D. J. Eisenstein, K. T. Mehta, and A. J. Cuesta, Mon. Not. Roy. Astron. Soc.427, 2146 (2012), arXiv:1202.0091 [astro-ph.CO]
Pith/arXiv arXiv 2012
-
[41]
Y .-Z. Ma, G.-D. Gong, N. Sui, and P. He, Mon. Not. Roy. As- tron. Soc.475, 379 (2018), arXiv:1711.08756 [astro-ph.CO]
Pith/arXiv arXiv 2018
-
[42]
R. J. Turner and C. Blake, Mon. Not. Roy. Astron. Soc.526, 337 (2023), arXiv:2306.16664 [astro-ph.CO]
arXiv 2023
-
[43]
A. J. Tishue, C. Shiveshwarkar, and G. Holder, (2025), arXiv:2510.25821 [astro-ph.CO]
arXiv 2025
-
[44]
Suresh, H
R. Suresh, H. Bishnoi, A. V . Kuklin, A. Parikh, M. Molokeev, R. Harinarayanan, S. Gharat, and P. Hiba, Front. in Phys.12, 1322162 (2024)
2024
-
[45]
Y . Gong and R. Bean, Phys. Rev. D112, 023527 (2025), arXiv:2505.12720 [astro-ph.CO]
arXiv 2025
-
[46]
O. Ronneberger, P. Fischer, and T. Brox, arXiv e-prints , arXiv:1505.04597 (2015), arXiv:1505.04597 [cs.CV]
Pith/arXiv arXiv 2015
-
[47]
Y . Wang and X. Yang, Astrophys. J.969, 76 (2024), arXiv:2406.14101 [astro-ph.IM]
arXiv 2024
-
[48]
X. Xiao, J. Ding, X. Luo, S. K. Lan, L. Xiao, S. Liu, X. Wang, L. Zhang, and X.-D. Li, Astrophys. J.994, 204 (2025), arXiv:2411.11280 [astro-ph.CO]
arXiv 2025
-
[49]
A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, J. Uszkoreit, and N. Houlsby, arXiv e-prints , arXiv:2010.11929 (2020), arXiv:2010.11929 [cs.CV]
Pith/arXiv arXiv 2010
-
[50]
Z. Liu, Y . Lin, Y . Cao, H. Hu, Y . Wei, Z. Zhang, S. Lin, and B. Guo, arXiv e-prints , arXiv:2103.14030 (2021), arXiv:2103.14030 [cs.CV]
Pith/arXiv arXiv 2021
-
[51]
C. Li, Y . Zhang, C. Cui, D. Fan, Y . Zhao, X.-B. Wu, B. He, Y . Xu, S. Li, J. Han, Y . Tao, L. Mi, H. Yang, and S. Yang, Monthly Notices of the Royal Astronomical Society 10.1093/mnras/stab1650 (2021)
-
[52]
N. Sahakyan, V . Vardanyan, and M. Khachatryan, Mon. Not. Roy. Astron. Soc.519, 3000 (2022), arXiv:2212.06614 [astro- ph.HE]
arXiv 2022
-
[53]
A. Tolamatti, K. K. Singh, and K. K. Yadav, Mon. Not. Roy. As- tron. Soc.523, 5341 (2023), arXiv:2306.15393 [astro-ph.HE]
arXiv 2023
-
[54]
J. Coronado-Blázquez, Mon. Not. Roy. Astron. Soc.521, 4156 (2023), arXiv:2303.07922 [astro-ph.HE]
arXiv 2023
-
[55]
Donoso-Oliva, I
C. Donoso-Oliva, I. Becker, P. Protopapas, G. Cabrera-Vives, M. Vishnu, and H. Vardhan, Astron. Astrophys.670, A54 (2023)
2023
-
[56]
H. W. Leung and J. Bovy, Monthly Notices of the Royal Astro- nomical Society527, 1494–1520 (2023)
2023
-
[57]
Y . Gong and R. Bean, Phys. Rev. D109, 123525 (2024), arXiv:2403.04664 [astro-ph.CO]
arXiv 2024
- [58]
-
[59]
J. H. Friedman, The Annals of Statistics29, 1189 (2001)
2001
-
[60]
G. Ke, Q. Meng, T. Finley, T. Wang, W. Chen, W. Ma, Q. Ye, and T.-Y . Liu, in Advances in Neural Information Processing Systems, V ol. 30 (2017)
2017
-
[61]
Vaswani, N
A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, in Advances in Neural Information Processing Systems, V ol. 30 (2017)
2017
-
[62]
N. A. Maksimova, L. H. Garrison, D. J. Eisenstein, B. Hadzhiyska, S. Bose, and T. P. Satterthwaite, Mon. Not. Roy. Astron. Soc.508, 4017 (2021), arXiv:2110.11398 [astro- ph.CO]
arXiv 2021
-
[63]
L. H. Garrison, D. J. Eisenstein, and P. A. Pinto, Mon. Not. Roy. Astron. Soc.485, 3370 (2019), arXiv:1810.02916 [astro- ph.CO]
Pith/arXiv arXiv 2019
-
[64]
L. H. Garrison, D. J. Eisenstein, D. Ferrer, N. A. Maksimova, and P. A. Pinto, Mon. Not. Roy. Astron. Soc.508, 575 (2021), arXiv:2110.11392 [astro-ph.CO]
arXiv 2021
-
[65]
N. Aghanim et al. (Planck), Astron. Astrophys.641, A6 (2020), [Erratum: Astron.Astrophys. 652, C4 (2021)], arXiv:1807.06209 [astro-ph.CO]
Pith/arXiv arXiv 2020
-
[66]
S. Yuan, L. H. Garrison, B. Hadzhiyska, S. Bose, and D. J. Eisenstein, Mon. Not. Roy. Astron. Soc.510, 3301 (2022), arXiv:2110.11412 [astro-ph.CO]
arXiv 2022
-
[67]
S. Yuan et al., Mon. Not. Roy. Astron. Soc.530, 947 (2024), arXiv:2306.06314 [astro-ph.CO]
arXiv 2024
-
[68]
Z. Zheng, A. L. Coil, and I. Zehavi, Astrophys. J.667, 760 (2007), arXiv:astro-ph/0703457
Pith/arXiv arXiv 2007
-
[69]
Hadzhiyska et al., (2023), arXiv:2305.11935 [astro-ph.CO]
B. Hadzhiyska et al., (2023), arXiv:2305.11935 [astro-ph.CO]
arXiv 2023
-
[70]
J. L. Tinker, B. E. Robertson, A. V . Kravtsov, A. Klypin, M. S. Warren, G. Yepes, and S. Gottlöber, Astrophys. J.724, 878 (2010), arXiv:1001.3162 [astro-ph.CO]
Pith/arXiv arXiv 2010
-
[71]
B. Hadzhiyska, L. H. Garrison, D. Eisenstein, and S. Bose, Mon. Not. Roy. Astron. Soc.509, 2194 (2021), arXiv:2110.11413 [astro-ph.CO]
arXiv 2021
-
[72]
V . Desjacques, D. Jeong, and F. Schmidt, Phys. Rept.733, 1 (2018), arXiv:1611.09787 [astro-ph.CO]
Pith/arXiv arXiv 2018
-
[73]
S. C. Hotinli, K. M. Smith, M. S. Madhavacheril, and M. Kamionkowski, Phys. Rev. D104, 083529 (2021), arXiv:2108.02207 [astro-ph.CO]
arXiv 2021
-
[74]
S. C. Hotinli, E. Pierpaoli, S. Ferraro, and K. Smith, Phys. Rev. D108, 083508 (2023), arXiv:2305.15462 [astro-ph.CO]
arXiv 2023
-
[75]
B. Soergel, A. Saro, T. Giannantonio, G. Efstathiou, and K. Dolag, Mon. Not. Roy. Astron. Soc.478, 5320 (2018), arXiv:1712.05714 [astro-ph.CO]
Pith/arXiv arXiv 2018
-
[76]
B. Hadzhiyska, S. Ferraro, R. Pakmor, S. Bose, A. M. Delgado, C. Hernández-Aguayo, R. Kannan, V . Springel, S. D. M. White, and L. Hernquist, Mon. Not. Roy. Astron. Soc.526, 369 (2023), arXiv:2305.00992 [astro-ph.CO]
arXiv 2023
-
[77]
P. G. Ferreira, R. Juszkiewicz, H. A. Feldman, M. Davis, and A. H. Jaffe, Astrophys. J. Lett.515, L1 (1999), arXiv:astro- ph/9812456
arXiv 1999
-
[78]
J. E. Forero-Romero, Y . Hoffman, S. Gottloeber, A. Klypin, and G. Yepes, Mon. Not. Roy. Astron. Soc.396, 1815 (2009), arXiv:0809.4135 [astro-ph]
Pith/arXiv arXiv 2009
-
[79]
L. Lucie-Smith, H. V . Peiris, and A. Pontzen, Mon. Not. Roy. Astron. Soc.490, 331 (2019), arXiv:1906.06339 [astro-ph.CO]
arXiv 2019
-
[80]
Y . Wanget al., Mon. Not. Roy. Astron. Soc.409, 737 (2010), arXiv:1006.3517 [astro-ph.CO]
Pith/arXiv arXiv 2010
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.