REVIEW 3 major objections 4 minor 102 references
AGILE builds an end-to-end LSST simulation of AGNs, galaxies, and stars so that selection methods and pipeline flux recovery can be tested against a known truth catalog before the survey flies.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · grok-4.5
2026-07-14 00:12 UTC pith:PEK6ZO2V
load-bearing objection Useful end-to-end LSST AGN mock and public DR1 test-bench; the empirical recipe is a known, stated limitation, not a hidden flaw. the 3 major comments →
AGILE: an end-to-end Rubin-LSST simulation of AGNs, galaxies, and stars I. Software description and first data release
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
AGILE supplies a complete 24 deg² truth catalog of AGNs, galaxies, and stars (0.2 < z < 5.5, log M/M⊙ > 8.5 for galaxies/AGNs, r < 27.5 for stars) and a 1 deg² three-year LSST-like image simulation (DR1) that quantifies Science Pipelines flux recovery and the completeness and purity of standard color-color and variability selections for Type 1 AGNs.
What carries the argument
The AGN recipe: complete galaxy samples are populated according to the observed accretion-rate distribution; each AGN receives an empirical optical/UV SED; optical light curves are generated with a damped random walk whose parameters are linked to the AGN’s physical properties; the resulting sources are rendered into LSST-like images and data products.
Load-bearing premise
That an AGN population built from existing empirical accretion-rate, SED, and damped-random-walk relations is close enough to the real sky that the completeness, purity, and flux-recovery numbers measured in the simulation will transfer to actual LSST data.
What would settle it
Compare AGILE DR1 selection completeness and purity, and Science Pipelines flux residuals, against the same statistics measured on real early LSST or precursor imaging of the COSMOS field once those data exist; a large systematic mismatch would falsify the claim that the empirical recipe is sufficiently faithful.
If this is right
- LSST AGN selection pipelines can be tuned and systematics quantified against a known truth catalog before survey data arrive.
- Flux-recovery biases measured for AGNs, galaxies, and stars in DR1 can be folded into photometric and variability selection forecasts.
- The same framework can be extended to Euclid-like NIR imaging for joint optical–NIR AGN selection tests.
- Public release of AGILE DR1 lets independent groups benchmark alternative classifiers and variability metrics on identical inputs.
Where Pith is reading between the lines
- If the damped-random-walk parameters prove too simple, later data releases may need more flexible stochastic models before purity numbers can be trusted for rare high-redshift AGNs.
- The 24 deg² truth catalog is large enough that the same machinery could be reused to forecast AGN number densities and contamination rates for other wide-field surveys beyond LSST and Euclid.
- Once real LSST light curves exist, residuals between observed and AGILE variability statistics will directly test how well empirical accretion-rate distributions capture the true population.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript presents AGILE, an end-to-end LSST simulation framework (INAF LSST in-kind) that injects AGNs into existing galaxy and star simulations using an empirical recipe: galaxies are populated according to the observed AGN accretion-rate distribution, each AGN receives an optical/UV SED, and optical variability is added via a damped random walk tied to AGN physical parameters. The authors construct a 24 deg² complete mock truth catalog (0.2 < z < 5.5, log M/M⊙ > 8.5 for AGNs/galaxies, r < 27.5 for stars) and a pilot AGILE DR1 (1 deg² COSMOS-like field, three-year LSST strategy) that produces LSST-like images and data products. DR1 is used to quantify LSST Science Pipelines flux recovery for AGNs, galaxies, and stars, and Type-1 AGN completeness/purity under typical color–color and variability selections; the DR1 dataset is released as a public test-bench.
Significance. If the software, truth catalog, and DR1 products are as described, this is a useful community infrastructure paper for LSST AGN selection, classification, and systematics studies. End-to-end mocks that couple accretion-rate demographics, SEDs, physically linked DRW variability, and LSST-like imaging are scarce; a shared 1 deg² three-year pilot with pipeline flux-recovery and selection metrics would be a practical test-bench for method development ahead of survey operations. The contribution is primarily methodological and data-release oriented rather than a new physical result; its value rests on transparency of the recipe, reproducibility of the products, and clarity of the reported completeness/purity and flux-recovery numbers.
major comments (3)
- The abstract asserts that AGILE DR1 quantifies LSST Science Pipelines flux recovery and Type-1 completeness/purity under color–color and variability selections, but does not report the actual metrics, uncertainties, selection definitions, or magnitude/redshift dependence. Those quantitative results are load-bearing for the pilot’s scientific usefulness; the full text must present them with clear truth-matching criteria, purity/completeness definitions, and comparison to injected truth so readers can judge whether DR1 is an adequate test-bench.
- The AGN recipe is built from empirical relations (accretion-rate distribution, SED assignment, DRW parameters linked to physical quantities). For a methods/data-release paper this is acceptable, but the manuscript must state explicitly which observational relations and parameterizations are used, their domain of validity (z, mass, luminosity), and any known biases relative to real Type-1 populations. Without that documentation, users cannot assess when DR1 selection metrics will or will not transfer to on-sky LSST performance.
- The pilot is 1 deg² over three years in a COSMOS-like field. The paper should justify that this depth, area, and cadence sampling are sufficient for the claimed pipeline and selection tests (e.g., variability selection needs adequate light-curve sampling), and should discuss cosmic variance / field choice relative to the full 24 deg² truth catalog so that DR1 metrics are not over-interpreted as survey-wide forecasts.
minor comments (4)
- Abstract: report at least headline flux-recovery accuracy and Type-1 completeness/purity numbers (with rough uncertainties) so the pilot’s outcome is visible without opening the full data products.
- Clarify how AGILE interfaces with the existing galaxy and star simulations it is based on (which codes/catalogs, which quantities are inherited vs. overwritten by the AGN recipe).
- State data-access details for AGILE DR1 (format, truth tables, image products, pipeline version) in a dedicated data-availability statement suitable for a first data release.
- Briefly situate AGILE relative to other public LSST-oriented AGN or multi-object mocks so readers know what is new (physically linked DRW, end-to-end images, joint AGN–galaxy–star truth).
Circularity Check
No significant circularity: AGILE is a forward mock-catalog test-bench that injects truth by construction and scores pipelines/selections against it.
full rationale
From the abstract and stated goals, AGILE builds a complete mock truth catalog by populating existing galaxy/star simulations with AGNs via an empirical recipe (observed accretion-rate distribution, optical/UV SEDs, DRW variability tied to physical parameters), then produces LSST-like images and data products. AGILE DR1 is used to measure flux recovery of the LSST Science Pipelines and Type-1 AGN completeness/purity under color-color and variability cuts, with the dataset shared as a public test-bench. Measuring recovery against injected truth is the intended methodology of an end-to-end simulation, not a self-definitional or fitted-input-called-prediction loop that claims an independent physical result. The empirical AGN recipe is an explicit modeling input, not a quantity derived from the same DR1 selection metrics later reported. No load-bearing uniqueness theorem, self-citation chain, or renaming of a known result is present in the available claims. Mild transferability risk (whether the empirical recipe is faithful enough for metrics to apply on-sky) is a standard simulation limitation, not circularity of the derivation chain.
Axiom & Free-Parameter Ledger
free parameters (4)
- AGN accretion-rate distribution (empirical)
- Optical/UV SED template assignment
- Damped random walk variability parameters linked to AGN physics
- Mass, redshift, and magnitude cuts of the truth catalog
axioms (4)
- domain assumption Existing galaxy and star simulations are an adequate base population for LSST-depth AGN science mocks.
- domain assumption Observed AGN accretion-rate distributions can be used to populate complete galaxy samples at 0.2<z<5.5.
- domain assumption A damped random walk with parameters connected to AGN physical quantities is a sufficient model of optical AGN variability for LSST selection tests.
- ad hoc to paper LSST survey strategy over three years in a 1 deg² COSMOS-like field is representative enough for pilot pipeline and selection metrics.
invented entities (2)
-
AGILE simulation software and AGN recipe
no independent evidence
-
AGILE 24 deg² truth catalog and AGILE DR1 (1 deg², 3 yr)
no independent evidence
read the original abstract
Contemporary large-scale surveys such as the Vera C. Rubin Observatory Legacy Survey of Space and Time (LSST) and Euclid present an unprecedented discovery potential for studying AGNs at the population level in the big data era. However, one major challenge is the accurate identification and classification of AGNs from optical/NIR photometry, or variability data alone. In order to optimize AGN selection, classification, and systematics, as well as to test different data analysis tools, we present AGILE (AGNs In the LSST Era), an LSST end-to-end simulation software. AGILE -- developed as part of the INAF LSST in-kind contribution -- is capable of simulating the anticipated AGN population in LSST and Euclid. We based AGILE on existing simulations of galaxies and stars, while we developed an AGN recipe based on empirical relations. AGILE populates complete galaxy samples with AGNs according to the observed AGN accretion rate distribution, and each AGN is assigned an optical/UV spectral energy distribution. Optical AGN variability is added using a damped random walk model connected to the AGN physical parameters. Finally, AGILE creates both LSST-like images and related data products. Using AGILE, we build a $24$ deg$^2$ complete mock truth catalog of AGNs, galaxies, and stars with $0.2 < z < 5.5$, $\log M/M_\odot > 8.5$ (AGNs and galaxies), and $r < 27.5$ mag (stars). We perform a pilot simulation (AGILE DR1) consisting of $1$ deg$^2$ of LSST operations in the COSMOS field observed up to three years according to the survey strategy. We use AGILE DR1 to quantify the accuracy of the LSST Science Pipelines in recovering true fluxes of AGNs, galaxies, and stars. We quantify the LSST completeness and purity in recovering Type 1 AGNs using typical color-color and variability selections. We share the AGILE DR1 dataset, an ideal test-bench for further scientific exploitation.
Reference graph
Works this paper leans on
-
[1]
D. Baumann, inTheoretical Advanced Study Institute in Elementary Particle Physics: Physics of the Large and the Small(2011) pp. 523–686, arXiv:0907.5424 [hep-th]
Pith/arXiv arXiv 2011
-
[2]
Ach´ ucarroet al., (2022), arXiv:2203.08128 [astro- ph.CO]
A. Ach´ ucarroet al., (2022), arXiv:2203.08128 [astro- ph.CO]
Pith/arXiv arXiv 2022
-
[3]
X. Chen, Adv. Astron.����, 638979 (2010), arXiv:1002.1416 [astro-ph.CO]
Pith/arXiv arXiv 2010
-
[4]
Y. Wang, Commun. Theor. Phys.��, 109 (2014), arXiv:1303.1523 [hep-th]
Pith/arXiv arXiv 2014
-
[5]
Green, PoS��������, 005 (2024), arXiv:2212.08685 [hep-ph]
D. Green, PoS��������, 005 (2024), arXiv:2212.08685 [hep-ph]
Pith/arXiv arXiv 2024
-
[6]
P. A. R. Adeet al.(BICEP, Keck), Phys. Rev. Lett. ���, 151301 (2021), arXiv:2110.00483 [astro-ph.CO]
arXiv 2021
-
[7]
X. Chen and Y. Wang, Phys. Rev. D��, 063511 (2010), arXiv:0909.0496 [astro-ph.CO]
Pith/arXiv arXiv 2010
- [8]
-
[9]
D. Baumann and D. Green, Phys. Rev. D��, 103520 (2012), arXiv:1109.0292 [hep-th]
Pith/arXiv arXiv 2012
- [10]
- [11]
-
[12]
J.-O. Gong, S. Pi, and M. Sasaki, JCAP��, 043 (2013), arXiv:1306.3691 [hep-th]
Pith/arXiv arXiv 2013
-
[13]
N. Arkani-Hamed and J. Maldacena, (2015), arXiv:1503.08043 [hep-th]
Pith/arXiv arXiv 2015
-
[14]
D. Baumann and D. Green, JCAP��, 014 (2011), arXiv:1102.5343 [hep-th]
Pith/arXiv arXiv 2011
-
[15]
V. Assassi, D. Baumann, D. Green, and L. McAllister, JCAP��, 033 (2014), arXiv:1304.5226 [hep-th]
Pith/arXiv arXiv 2014
-
[16]
N. Craig and D. Green, JHEP��, 102 (2014), arXiv:1403.7193 [hep-ph]
Pith/arXiv arXiv 2014
-
[17]
E. Dimastrogiovanni, M. Fasiello, and M. Kamionkowski, JCAP��, 017 (2016), arXiv:1504.05993 [astro-ph.CO]
Pith/arXiv arXiv 2016
-
[18]
H. Lee, D. Baumann, and G. L. Pimentel, JHEP��, 040 (2016), arXiv:1607.03735 [hep-th]
Pith/arXiv arXiv 2016
-
[19]
P. D. Meerburg, M. M¨ unchmeyer, J. B. Mu˜ noz, and X. Chen, JCAP��, 050 (2017), arXiv:1610.06559 [astro- ph.CO]
Pith/arXiv arXiv 2017
-
[20]
X. Chen, Y. Wang, and Z.-Z. Xianyu, Phys. Rev. Lett. ���, 261302 (2017), arXiv:1610.06597 [hep-th]
Pith/arXiv arXiv 2017
-
[21]
X. Chen, Y. Wang, and Z.-Z. Xianyu, JHEP��, 051 (2016), arXiv:1604.07841 [hep-th]
Pith/arXiv arXiv 2016
-
[22]
X. Chen, Y. Wang, and Z.-Z. Xianyu, JHEP��, 058 (2017), arXiv:1612.08122 [hep-th]
Pith/arXiv arXiv 2017
-
[23]
H. An, M. McAneny, A. K. Ridgway, and M. B. Wise, JHEP��, 105 (2018), arXiv:1706.09971 [hep-ph]
Pith/arXiv arXiv 2018
-
[24]
X. Chen, Y. Wang, and Z.-Z. Xianyu, JCAP��, 006 (2017), arXiv:1703.10166 [hep-th]
Pith/arXiv arXiv 2017
-
[25]
S. Kumar and R. Sundrum, JHEP��, 011 (2018), arXiv:1711.03988 [hep-ph]
Pith/arXiv arXiv 2018
-
[26]
D. Baumann, G. Goon, H. Lee, and G. L. Pimentel, JHEP��, 140 (2018), arXiv:1712.06624 [hep-th]
Pith/arXiv arXiv 2018
-
[27]
X. Chen, Y. Wang, and Z.-Z. Xianyu, JHEP��, 022 (2018), arXiv:1805.02656 [hep-ph]
Pith/arXiv arXiv 2018
-
[28]
S. Kumar and R. Sundrum, JHEP��, 120 (2019), arXiv:1811.11200 [hep-ph]
Pith/arXiv arXiv 2019
-
[29]
Wu, JHEP��, 125 (2019), arXiv:1812.10654 [hep- ph]
Y.-P. Wu, JHEP��, 125 (2019), arXiv:1812.10654 [hep- ph]
Pith/arXiv arXiv 2019
-
[30]
E. Dimastrogiovanni, M. Fasiello, and G. Tasinato, JCAP��, 016 (2018), arXiv:1806.00850 [astro-ph.CO]
Pith/arXiv arXiv 2018
-
[31]
S. Lu, Y. Wang, and Z.-Z. Xianyu, JHEP��, 011 (2020), arXiv:1907.07390 [hep-th]
Pith/arXiv arXiv 2020
-
[32]
A. Hook, J. Huang, and D. Racco, JHEP��, 105 (2020), arXiv:1907.10624 [hep-ph]
Pith/arXiv arXiv 2020
-
[33]
A. Hook, J. Huang, and D. Racco, Phys. Rev. D���, 023519 (2020), arXiv:1908.00019 [hep-ph]
Pith/arXiv arXiv 2020
-
[34]
S. Kumar and R. Sundrum, JHEP��, 077 (2020), arXiv:1908.11378 [hep-ph]
Pith/arXiv arXiv 2020
-
[35]
L.-T. Wang and Z.-Z. Xianyu, JHEP��, 044 (2020), arXiv:1910.12876 [hep-ph]
Pith/arXiv arXiv 2020
-
[36]
L. Li, T. Nakama, C. M. Sou, Y. Wang, and S. Zhou, JHEP��, 067 (2019), arXiv:1903.08842 [astro-ph.CO]
Pith/arXiv arXiv 2019
-
[37]
S. Alexander, S. J. Gates, L. Jenks, K. Koutrolikos, and 9 E. McDonough, JHEP��, 156 (2019), arXiv:1907.05829 [hep-th]
Pith/arXiv arXiv 2019
-
[38]
A. Bodas, S. Kumar, and R. Sundrum, JHEP��, 079 (2021), arXiv:2010.04727 [hep-ph]
Pith/arXiv arXiv 2021
-
[39]
L.-T. Wang and Z.-Z. Xianyu, JHEP��, 082 (2020), arXiv:2004.02887 [hep-ph]
Pith/arXiv arXiv 2020
-
[40]
Lu, JHEP��, 157 (2022), arXiv:2103.05958 [hep-th]
S. Lu, JHEP��, 157 (2022), arXiv:2103.05958 [hep-th]
Pith/arXiv arXiv 2022
-
[41]
Q. Lu, M. Reece, and Z.-Z. Xianyu, JHEP��, 098 (2021), arXiv:2108.11385 [hep-ph]
Pith/arXiv arXiv 2021
-
[42]
E. Dimastrogiovanni, M. Fasiello, and A. E. G¨ umr¨ ukc¨ uogl¨ u, JCAP��, 047 (2021), arXiv:2108.06722 [astro-ph.CO]
Pith/arXiv arXiv 2021
-
[43]
Y. Cui and Z.-Z. Xianyu, Phys. Rev. Lett.���, 111301 (2022), arXiv:2112.10793 [hep-ph]
Pith/arXiv arXiv 2022
-
[44]
X. Tong and Z.-Z. Xianyu, JHEP��, 194 (2022), arXiv:2203.06349 [hep-ph]
Pith/arXiv arXiv 2022
-
[45]
Z. Qin and Z.-Z. Xianyu, JHEP��, 192 (2022), arXiv:2205.01692 [hep-th]
Pith/arXiv arXiv 2022
-
[46]
M. Reece, L.-T. Wang, and Z.-Z. Xianyu, Phys. Rev. D���, L101304 (2023), arXiv:2204.11869 [hep-ph]
Pith/arXiv arXiv 2023
-
[47]
X. Chen, R. Ebadi, and S. Kumar, JCAP��, 083 (2022), arXiv:2205.01107 [hep-ph]
Pith/arXiv arXiv 2022
-
[48]
N. Maru and A. Okawa, Int. J. Mod. Phys. A��, 2350075 (2023), arXiv:2206.06651 [hep-ph]
Pith/arXiv arXiv 2023
-
[49]
X. Chen, J. Fan, and L. Li, (2023), arXiv:2303.03406 [hep-ph]
Pith/arXiv arXiv 2023
-
[50]
N. Craig, S. Kumar, and A. McCune, JHEP��, 108 (2024), arXiv:2401.10976 [hep-ph]
Pith/arXiv arXiv 2024
-
[51]
J. Quintin, X. Chen, and R. Ebadi, JCAP��, 026 (2024), arXiv:2405.11016 [astro-ph.CO]
Pith/arXiv arXiv 2024
-
[52]
A. Bodas, E. Broadberry, and R. Sundrum, JHEP��, 115 (2025), arXiv:2409.07524 [hep-ph]
Pith/arXiv arXiv 2025
-
[53]
J. Hubisz, S. J. Lee, H. Li, and B. Sambasivam, Phys. Rev. D���, 023543 (2025), arXiv:2408.08951 [astro- ph.CO]
Pith/arXiv arXiv 2025
-
[54]
Chakraborty, (2025), arXiv:2501.07672 [hep-th]
P. Chakraborty, (2025), arXiv:2501.07672 [hep-th]
arXiv 2025
-
[55]
C. de Rham, S. Jazayeri, and A. J. Tolley, Phys. Rev. D���, 083531 (2025), arXiv:2506.19198 [hep-th]
arXiv 2025
- [56]
-
[57]
P. Chakraborty, T. Cohen, D. Green, and Y. Huang, (2025), arXiv:2508.08359 [hep-th]
Pith/arXiv arXiv 2025
- [58]
- [59]
-
[60]
Y. Jiang, G. L. Pimentel, and C. Yang, (2025), arXiv:2512.23796 [hep-th]
Pith/arXiv arXiv 2025
-
[61]
Y. Akramiet al.(Planck), Astron. Astrophys.���, A9 (2020), arXiv:1905.05697 [astro-ph.CO]
Pith/arXiv arXiv 2020
-
[62]
G. Jung, M. Citran, B. van Tent, L. Dumilly, and N. Aghanim, Astron. Astrophys.���, A204 (2025), arXiv:2504.00884 [astro-ph.CO]
arXiv 2025
-
[63]
Chaussidonet al., JCAP��, 029 (2025), arXiv:2411.17623 [astro-ph.CO]
E. Chaussidonet al., JCAP��, 029 (2025), arXiv:2411.17623 [astro-ph.CO]
Pith/arXiv arXiv 2025
-
[64]
Dor´ eet al.(SPHEREx), (2014), arXiv:1412.4872 [astro-ph.CO]
O. Dor´ eet al.(SPHEREx), (2014), arXiv:1412.4872 [astro-ph.CO]
Pith/arXiv arXiv 2014
-
[65]
Adeet al.(Simons Observatory), JCAP��, 056 (2019), arXiv:1808.07445 [astro-ph.CO]
P. Adeet al.(Simons Observatory), JCAP��, 056 (2019), arXiv:1808.07445 [astro-ph.CO]
Pith/arXiv arXiv 2019
-
[66]
Y. Mellieret al.(Euclid), Astron. Astrophys.���, A1 (2025), arXiv:2405.13491 [astro-ph.CO]
arXiv 2025
-
[67]
N. Arkani-Hamed, D. Baumann, H. Lee, and G. L. Pimentel, JHEP��, 105 (2020), arXiv:1811.00024 [hep- th]
Pith/arXiv arXiv 2020
-
[68]
D. Baumann, C. Duaso Pueyo, A. Joyce, H. Lee, and G. L. Pimentel, JHEP��, 204 (2020), arXiv:1910.14051 [hep-th]
Pith/arXiv arXiv 2020
-
[69]
G. L. Pimentel and D.-G. Wang, JHEP��, 177 (2022), arXiv:2205.00013 [hep-th]
Pith/arXiv arXiv 2022
-
[70]
S. Jazayeri and S. Renaux-Petel, JHEP��, 137 (2022), arXiv:2205.10340 [hep-th]
Pith/arXiv arXiv 2022
-
[71]
Z. Qin and Z.-Z. Xianyu, JHEP��, 059 (2023), arXiv:2208.13790 [hep-th]
Pith/arXiv arXiv 2023
-
[72]
Z. Qin and Z.-Z. Xianyu, JHEP��, 001 (2023), arXiv:2301.07047 [hep-th]
Pith/arXiv arXiv 2023
-
[73]
S. Aoki, T. Noumi, F. Sano, and M. Yamaguchi, JHEP ��, 073 (2024), arXiv:2312.09642 [hep-th]
Pith/arXiv arXiv 2024
-
[74]
S. Aoki, L. Pinol, F. Sano, M. Yamaguchi, and Y. Zhu, JHEP��, 176 (2024), arXiv:2404.09547 [hep-th]
Pith/arXiv arXiv 2024
-
[75]
J. Chen, B. Feng, and Y.-X. Tao, JHEP��, 075 (2025), arXiv:2411.03088 [hep-th]
Pith/arXiv arXiv 2025
-
[76]
H. Liu and Z.-Z. Xianyu, JHEP��, 183 (2025), arXiv:2412.07843 [hep-th]
Pith/arXiv arXiv 2025
-
[77]
L.-T. Wang, Z.-Z. Xianyu, and Y.-M. Zhong, JHEP��, 085 (2022), arXiv:2109.14635 [hep-ph]
Pith/arXiv arXiv 2022
-
[78]
D. Werth, L. Pinol, and S. Renaux-Petel, Phys. Rev. Lett.���, 141002 (2024), arXiv:2302.00655 [hep-th]
Pith/arXiv arXiv 2024
-
[79]
L. Pinol, S. Renaux-Petel, and D. Werth, JCAP��, 019 (2025), arXiv:2312.06559 [astro-ph.CO]
Pith/arXiv arXiv 2025
-
[80]
D. Werth, L. Pinol, and S. Renaux-Petel, Class. Quant. Grav.��, 175015 (2024), arXiv:2402.03693 [astro-ph.CO]
Pith/arXiv arXiv 2024
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.