REVIEW 3 major objections 4 minor 51 references
Highly Variable Quasar Candidates Selected from 4XMM-DR13 with Machine Learning
T0 review · 3 major / 4 minor · reviewed 2026-08-10 · deepseek-v4-flash
Pith's one-line read This paper claims to identify 12 quasar candidates in 4XMM-DR13 whose soft X-ray flux changed by at least a factor of 10 between XMM-Newton epochs, extending extreme X-ray variability to optically faint quasars around r ~ 22, and argues…
desk verdict Useful candidate sample, but the ROSAT-based rarity claim doesn't survive contact with the flux limits. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The random forest classifier uses 15 features: X-ray flux in 0.2-12 keV, four XMM hardness ratios (hr1-hr4), X-ray-to-optical flux ratio, SDSS r magnitude and colors (u-g, g-r, r-i, i-z), WISE magnitudes W1 and W2, and infrared colors (z-W1, W1-W2). SMOTE oversampling balances the training set of 23,501 spectroscopically classified sources (16,826 quasars, 3,800 stars, 2,875 galaxies). Stellar contaminants are removed by a Gaia proper-motion probability cut using the bivariate normal of pmra and pmdec, and extreme variability is defined by the ratio of soft-band fluxes between XMM epochs exceeding 10.
What would settle it
Obtain optical spectra of the 12 candidates: if a majority lack broad emission lines or other quasar signatures, the selection is contaminated. Alternatively, a targeted search of deeper all-sky X-ray data (e.g., eROSITA) for optically faint quasars with >100-fold soft X-ray variability would directly test the claimed rarity.
Extended reading notes
Core claim
The central discovery is a sample of 12 quasar candidates in the 4XMM-DR13 catalog that each changed their 0.2-2 keV flux by a factor of at least 10 between XMM-Newton observations spanning roughly 20 years. The selection pipeline first classifies 100,183 X-ray sources with optical and mid-infrared counterparts into quasar, galaxy, and star candidates using random forest; after a Gaia proper-motion cut to remove stars and a FIRST cut to remove radio-loud objects, it isolates sources with at least two clean XMM detections, a soft-band flux ratio greater than 10, and a minimum flux of 5e-15 erg $cm^{-2}$ $s^{-1}$, followed by visual inspection of light curves and images. The 12 survivors have SDSS i-band magnitudes out to about 22, making them fainter than previously known highly variable quasars. The paper also reports that none of the 12 are detected in ROSAT, and given ROSAT's flux limit this implies that quasars with variability factors above 100 are extremely rare.
Load-bearing premise
The random forest classifier was trained on relatively bright, spectroscopically confirmed quasars, and none of the 12 final candidates has a confirming spectrum; if several are actually stars or galaxies, the sample and the rarity inference collapse.
Editorial extensions
If this is right
- The population of quasars with extreme soft X-ray variability (factor >=10) extends to i-band magnitude about 22, not just the bright AGNs previously catalogued.
- The ROSAT non-detections imply that variability amplitudes above 100 are very rare among these faint quasars, placing a constraint on models of changing accretion or obscuration.
- The random forest pipeline can be applied to future XMM-Newton data releases to enlarge the sample and refine the rarity estimate.
- The 12 candidates are prime targets for spectroscopy and multiwavelength monitoring to distinguish between changing accretion rate and obscuration as the driver.
Reading between the lines
- If any of the 12 turn out to be flaring M dwarfs or X-ray binaries, the rarity claim would need to be revised downward; this is testable by spectroscopy.
- The ROSAT non-detection may simply reflect that these quasars were in a low state during the 1990s; the true rate of >100-fold events could be higher than inferred if such events are short-lived.
- Because the training set is dominated by bright quasars, the classifier's precision at the faint end is uncertain; future spectroscopic follow-up of a random subset of the 52,486 candidates would quantify the contamination rate.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This manuscript applies a random forest classifier to 4XMM-DR13 sources cross-matched with SDSS DR18 and AllWISE photometry, using a training sample of spectroscopically classified stars, galaxies, and quasars. After Gaia proper-motion filtering and exclusion of FIRST counterparts, the authors select 12 quasar candidates that changed their 0.2–2 keV flux by a factor of at least 10 between XMM-Newton epochs, extending to r-band magnitudes around 22. The paper further reports that none of the 12 candidates is detected in ROSAT and concludes that quasars exhibiting variations of more than two orders of magnitude are extremely rare.
Significance. The sample construction is careful: the training set is built from SDSS and LAMOST spectra, the classifier is validated with cross-validation and external catalogs (SIMBAD, NED), light curves and X-ray images are visually inspected, and a comparison sample of known quasars is selected with identical criteria. The public machine-readable table and GitHub repository are useful assets. If spectroscopically confirmed, the 12 candidates would extend the known population of extreme soft-X-ray variable quasars to the optically faint end. However, two load-bearing issues remain: the ROSAT-based rarity inference is not supported by the data presented, and the reliability of the classifier for the faintest, uncataloged sources is not quantified. The paper itself acknowledges in Section 6 that spectra are needed to confirm the quasar nature of the candidates.
major comments (3)
- [Abstract; Section 5, Figures 5–6] The claim that ROSAT non-detections imply that quasars with variability factors of 100 or more are 'exceedingly rare' is not supported by the data. All 12 candidates have bright-state 0.2–2 keV fluxes below 1e-13 erg/s/cm2 (Table 5; the largest is 9.41e-14 for 4XMM J122809.3+435358), while the paper itself quotes the RASS sensitivity as a few times 10^-13 erg/s/cm2. Non-detection in RASS is therefore expected even if these objects are non-variable. Constraining the occurrence rate of >100x variability would require a sample whose bright-state fluxes are above the RASS limit, or a statistical model of RASS detection probability that accounts for the sensitivity and epoch. Please remove or substantially qualify this inference.
- [Section 4.2, Table 5] Table 5 reports the XMM-Newton and Chandra soft X-ray fluxes without uncertainties, and the text only states that candidates with 'large flux uncertainties' were rejected after visual inspection. Since the factor-of-10 variability claim is the central result, the reader needs the individual flux uncertainties (and ideally detection significances) for every epoch to verify that the flux ratios are statistically robust, especially for the faintest measurements near the 5e-15 erg/s/cm2 threshold.
- [Section 4.1, Figure 4, Section 6] The random forest is trained on spectroscopically confirmed sources that are systematically brighter in X-rays than the 12 candidates (Figure 4), and none of the 12 has a spectrum. The paper acknowledges that spectra are needed, but to support the claim that the sample extends the highly variable quasar population, it should provide a quantitative estimate of classifier performance in the flux/magnitude regime of the candidates—for example, flux-bin precision and recall on the held-out test set, or the random-forest probability P_QSO for each of the 12 objects, which is currently omitted from Table 5.
minor comments (4)
- [Section 4.2, last paragraph] The text refers to 'the relatively deeper observations in the 4XMM-DR3 catalog'; this should be 4XMM-DR13.
- [Section 4.1] The proper-motion cut is described as log(fPM0) ≤ −4 in the prose and as log(fPM0) < −4 in the following sentence; please harmonize the inequality.
- [Section 3.4] The sentence 'The optimal value is log 152 for max_features' is unclear; please specify whether this is log2(152) or another convention.
- [Section 2.1 and Section 3] The relation among the 100,183 XMM-WISE-SDSS sources, the 23,501 training sources, and the 76,682 sources actually classified by the random forest is implicit; a brief sentence stateing that only the unclassified remainder is subject to the classifier would improve clarity.
Circularity Check
No significant circularity: the variability selection is applied after an externally validated classifier and is not used to fit any model parameter.
full rationale
The paper's derivation chain is not circular. The random forest classifier is trained on spectroscopically labeled SDSS/LAMOST sources (Table 1) with features drawn from XMM, SDSS, and AllWISE (Table 2), and its performance is evaluated by 5-fold cross-validation and external catalog checks (SIMBAD and NED). The variability selection in Section 4.2 is applied after classification: the soft-band flux ratio greater than 10 and the 5e-15 flux threshold are selection criteria, not fitted parameters, and the 12 final candidates are not used to adjust any model component. The comparison sample of 43 variable quasars is selected from the training set with the same criteria, which supports a distributional comparison rather than a circular validation. The self-citations (Y. Zhang et al. 2021 for matching radii and RF feature choices; Y. Fu et al. 2021, 2024 for the Gaia proper-motion probability cut) are methodological and are not load-bearing for the central claim of discovering faint, highly variable quasar candidates. The RASS non-detection inference in Section 5 is statistically fragile because all 12 candidates have bright-state fluxes below the ROSAT sensitivity of about 1e-13 erg cm-2 s-1, but this is an evidential-support problem rather than a circularity: the ROSAT non-detection is not used to define, train, or fit the selection, and no equation in the paper reduces the rarity conclusion to an input by construction. Therefore no circular step can be quoted, and the paper is self-contained against external benchmarks.
Assumptions & free parameters
free parameters (5)
- Random forest max_features =
log 152 (as stated in text; ambiguous)
- Random forest n_estimators =
151
- Soft X-ray flux threshold =
5e-15 erg s^-1 cm^-2
- Variability ratio threshold =
10
- Gaia proper-motion cut =
log10(f_PM0) < -4
assumptions (4)
- domain assumption The random forest classifier trained on SDSS/LAMOST spectroscopically classified sources generalizes to all unclassified sources in the XMM-WISE-SDSS sample, including optically faint ones around r ~ 22.
- domain assumption The 4XMM-DR13 soft-band fluxes, derived assuming an absorbed power law with photon index 1.7 and N_H = 3e20 cm^-2, yield flux ratios accurate enough to establish factor-of-10 variability.
- domain assumption The training set of 3,800 stars, 2,875 galaxies, and 16,826 quasars is representative of the unclassified XMM-WISE-SDSS population.
- domain assumption The ROSAT All-Sky Survey would have detected any of the 12 objects if they had been in a bright state above the RASS flux limit during the survey era.
Cite this review
Pith. "Pith review of Highly Variable Quasar Candidates Selected from 4XMM-DR13 with Machine Learning." pith.science (2026). https://pith.science/paper/AUAZ5T2O
@misc{pith2026250115254,
author = {Pith},
title = {Pith review of: Highly Variable Quasar Candidates Selected from 4XMM-DR13 with Machine Learning},
year = {2026},
howpublished = {\url{https://pith.science/paper/AUAZ5T2O}},
note = {Machine review of arXiv:2501.15254}
}
abstract
We present a sample of 12 quasar candidates with highly variable soft X-ray emission from the 4th XMM-newton Serendipitous Source Catalog (4XMM-DR13) using random forest. We obtained optical to mid-IR photometric data for the 4XMM-DR13 sources by correlating the sample with the SDSS DR18 photometric database and the AllWISE database. By cross-matching this sample with known spectral catalogs from the SDSS and LAMOST surveys, we obtained a training data set containing stars, galaxies, and quasars. The random forest algorithm was trained to classify the XMM-WISE-SDSS sample. We further filtered the classified quasar candidates with $\it{Gaia}$ proper motion to remove stellar contaminants. Finally, 53,992 quasar candidates have been classified, with 10,210 known quasars matched in SIMBAD. The quasar candidates have systematically lower X-ray fluxes than quasars in the training set, which indicates the classifier is helpful to single out fainter quasars. We constructed a sample of 12 sources from these quasars candidates which changed their soft X-ray fluxes by a factor of 10 over $\sim$ 20 years in the 4XMM-newton survey. Our selected highly variable quasar candidates extend the quasar sample, characterized by extreme soft X-ray variability, to the optically faint end with magnitudes around $r \sim 22$. None of the 12 sources were detected in ROSAT observations. Given the flux limit of ROSAT, the result suggests that quasars exhibiting variations of more than two orders of magnitudes are extremely rare.
Figures
Figures from the paper (3 more)
Reference graph
Works this paper leans on
- [1]
- [2]
- [3]
- [4]
- [5]
- [6]
-
[7]
N., Evans, J
Evans, I. N., Evans, J. D., Martínez-Galarza, J. R., et al. 2024, ApJS, 274, 22
2024
- [8]
Show all 51 references
-
[9]
2021, ApJS, 254, 6
Fu, Y., Wu, X.-B., Yang, Q., et al. 2021, ApJS, 254, 6
2021
-
[10]
R., & Brandt, W
Gibson, R. R., & Brandt, W. 2012, ApJ, 746, 54
2012
-
[11]
2012, A&A, 544, A80
Gonzalez-Martin, O., & Vaughan, S. 2012, A&A, 544, A80
2012
-
[12]
2015, ApJL, 803, L28
Grupe, D., Komossa, S., & Saxton, R. 2015, ApJL, 803, L28
2015
-
[13]
Ho, T. K. 1995, in Proc. 3rd Int. Conf. on Document Analysis and Recognition, 1, IEEE (Piscataway, NJ: IEEE ), 278
1995
-
[14]
2020, ApJ, 895, 114
Huang, J., Luo, B., Du, P., et al. 2020, ApJ, 895, 114
2020
-
[15]
2021, MNRAS, 507, 687
Jana, A., Kumari, N., Nandi, P., et al. 2021, MNRAS, 507, 687
2021
-
[16]
M., & Davis, S
Jiang, Y.-F., Blaes, O., Stone, J. M., & Davis, S. W. 2019, ApJ, 885, 144
2019
-
[17]
2023, MNRAS, 518, 6065
Jin, C., Done, C., Ward, M., et al. 2023, MNRAS, 518, 6065
2023
-
[18]
2017, A&A, 607, L9
Krumpe, M., Husemann, B., Tremblay, G., et al. 2017, A&A, 607, L9
2017
-
[19]
Li, D., Starling, R. L. C., Saxton, R. D., Pan, H.-W., & Yuan, W. 2022, MNRAS, 512, 3858
2022
-
[20]
2020, MNRAS, 492, 2335
Liu, Z., Liu, H.-Y., Cheng, H., Qiao, E., & Yuan, W. 2020, MNRAS, 492, 2335
2020
-
[21]
L., Zhao, Y.-H., Zhao, G., et al
Luo, A. L., Zhao, Y.-H., Zhao, G., et al. 2015, RAA, 15, 1095
2015
-
[22]
2019, PatRe, 91, 216
Luque, A., Carrasco, A., Martín, A., & de las Heras, A. 2019, PatRe, 91, 216
2019
-
[23]
M., Koerding, E., Knigge, C., Uttley, P., & Fender, R
McHardy, I. M., Koerding, E., Knigge, C., Uttley, P., & Fender, R. 2006, Natur, 444, 730
2006
-
[24]
Y., Sunyaev, R., & Khorunzhev, G
Medvedev, P., Gilfanov, M., Sazonov, S. Y., Sunyaev, R., & Khorunzhev, G. 2022, AstL, 48, 735
2022
-
[25]
2021, A&A, 652, A150
Mehdipour, M., Kriss, G., Kaastra, J., et al. 2021, A&A, 652, A150
2021
-
[26]
2017, A&A, 599, A82
Middei, R., Vagnetti, F., Bianchi, S., et al. 2017, A&A, 599, A82
2017
-
[27]
2020, ApJL, 889, L37
Ni, Q., Brandt, W., Yi, W., et al. 2020, ApJL, 889, L37
2020
-
[28]
L., Alston, W
Parker, M. L., Alston, W. N., Härer, L., et al. 2021, MNRAS, 508, 1798
2021
-
[29]
2011, JMLR, 12, 2825
Pedregosa, F., Varoquaux, G., Gramfort, A., et al. 2011, JMLR, 12, 2825
2011
-
[30]
2019, ApJ, 884, 80
Reeves, J., & Braito, V. 2019, ApJ, 884, 80
2019
-
[31]
2020, ApJL, 898, L1
Ricci, C., Kara, E., Loewenstein, M., et al. 2020, ApJL, 898, L1
2020
-
[32]
2011, arXiv:1106.3507 Schlafly, E
Saxton, R., Read, A., Esquej, P., Miniutti, G., & Alvarez, E. 2011, arXiv:1106.3507 Schlafly, E. F., Meisner, A.M., & Green, G.M. 2019, ApJS, 240, 30S
2011 arXiv
-
[33]
L., et al
Strotjohann, N., Saxton, R., Starling, R. L., et al. 2016, A&A, 592, A74
2016
-
[34]
1980, A&A, 86, 121
Sunyaev, R., & Titarchuk, L. 1980, A&A, 86, 121
1980
-
[35]
Taylor, M. B. 2005, in ASP Conf. Ser. 347, Astronomical Data Analysis Software and Systems XIV, ed. P. Shopbell, M. Britton, & R. Ebert (San
2005
-
[36]
D., III, Brandt, W
Timlin, J. D., III, Brandt, W. N., Zhu, S., et al. 2020, MNRAS, 498, 4033
2020
-
[37]
J., et al
Trakhtenbrot, B., Ricci, C., Koss, M. J., et al. 2017, MNRAS, 470, 800
2017
-
[38]
2002, MNRAS, 332, 231
Uttley, P., McHardy, I., & Papadakis, I. 2002, MNRAS, 332, 231
2002
-
[39]
2000, yCat, 9029, 0
Voges, W., Aschenbach, B., Boller, T., et al. 2000, yCat, 9029, 0
2000
-
[40]
N., Luo, B., et al
Wang, S., Brandt, W. N., Luo, B., et al. 2024, ApJ, 974, 2
2024
-
[41]
2022, A&A, 657, A77
Wang, Y., Kaastra, J., Mehdipour, M., et al. 2022, A&A, 657, A77
2022
-
[42]
A., Coriat, M., Traulsen, I., et al
Webb, N. A., Coriat, M., Traulsen, I., et al. 2020, A&A, 641, A136
2020
-
[43]
2012, AJ, 144, 49
Wu, X.-B., Hao, G., Jia, Z., Zhang, Y., & Peng, N. 2012, AJ, 144, 49
2012
-
[44]
D., & Li, X
Xu, X.-j., Wang, Q. D., & Li, X. 2022, MNRAS, 516, 1263
2022
-
[45]
2022, ApJ, 941, 104
Yang, H., Hare, J., Kargaltsev, O., et al. 2022, ApJ, 941, 104
2022
- [46]
-
[47]
J., MacLeod, C
Yang, Q., Green, P. J., MacLeod, C. L., et al. 2023, ApJ, 953, 61
2023
-
[48]
G., Adelman, J., Anderson, J
York, D. G., Adelman, J., Anderson, J. E. J., et al. 2000, AJ, 120, 1579
2000
-
[49]
2014, ARA&A, 52, 529
Yuan, F., & Narayan, R. 2014, ARA&A, 52, 529
2014
-
[50]
2021, MNRAS, 503, 5263
Zhang, Y., Zhao, Y., & Wu, X.-B. 2021, MNRAS, 503, 5263
2021
-
[51]
2025, ApJ, 983, 36Z 12 The Astrophysical Journal, 985:23 (12pp), 2025 May 20 Wang et al
Zhang, Z., Luo, B., Jiang, L., et al. 2025, ApJ, 983, 36Z 12 The Astrophysical Journal, 985:23 (12pp), 2025 May 20 Wang et al
2025
Reviewed August 10, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.