Comparison of Photometric and Spectroscopic Labels in Classifying Dusty Stellar Sources Using Machine Learning in the Magellanic Clouds

Atefeh Javadi; Habib G. Khosroshahi; Iain McDonald; Jacco Th. van Loon; Joana Oliveira; Mahdi Abdollahi; Sepideh Ghaziasgar

REVIEW 3 major objections 3 minor 35 references

Comparison of Photometric and Spectroscopic Labels in Classifying Dusty Stellar Sources Using Machine Learning in the Magellanic Clouds

T0 review · 3 major / 3 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Photometric classification of dusty stars is reliable for oxygen-rich AGBs and young stellar objects, but confirms only 8% of red supergiant labels and under half of post-AGB labels.

desk verdict Useful question, plausible qualitative answer, but the quantitative confirmation rates in the abstract are not supported by the presented evidence. read the letter →

arxiv 2509.05531 v1 pith:OF2ZWM5Q submitted 2025-09-05 astro-ph.GA

Sepideh Ghaziasgar , Mahdi Abdollahi , Atefeh Javadi , Jacco Th. van Loon , Iain McDonald , Joana Oliveira , Habib G. Khosroshahi This is my paper · ORCID

classification astro-ph.GA

keywords dustystellarsourcesMagellanicCloudsphotometricclassificationspectroscopicProbabilisticRandomForestasymptoticgiantbranchstarsyoungobjectsredsupergiants

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This study asks whether photometric classifications of dusty stellar sources in the Magellanic Clouds can be trusted, by training a probabilistic random forest on spectroscopically labeled sources and applying it to a combined photometric catalog of more than 54,000 objects. The model confirms nearly all oxygen-rich AGB stars and about 95% of young stellar objects, but only about 8% of red supergiant labels and fewer than half of post-AGB labels survive. The authors conclude that photometry is reliable for abundant populations with distinctive infrared colors, while rare or overlapping classes require spectroscopic confirmation. If right, this gives survey astronomers a quantitative map of where photometric star classification can and cannot be used.

What carries the argument

The central machinery is a Probabilistic Random Forest (PRF), a supervised classifier that propagates uncertainties in both the input features and the training labels. It is trained on spectroscopically confirmed dusty stellar sources divided into five classes (YSO, CAGB, OAGB, RSG, PAGB) and then applied to a photometric catalog of 54,306 sources; the comparison between predicted and catalog labels is summarized in confusion matrices. The PRF's probabilistic outputs are what let the authors quote confirmation rates per class and identify which photometric labels are systematically reassigned.

What would settle it

Randomly select a few hundred photometric RSG and PAGB candidates from the 54,306-object catalog and obtain new infrared spectra. If the spectroscopic confirmation rate for RSGs is substantially higher than 8%, or for PAGBs above half, the paper's central claim about photometric unreliability for these classes is falsified; if it matches, the claim is supported.

Watch

Extended reading notes

Core claim

The paper's central claim is that photometric classification of dusty stellar sources is not uniformly reliable: it works for abundant classes with distinct infrared signatures (OAGBs and YSOs) and fails for rare or overlapping classes (RSGs and PAGBs). Using a Probabilistic Random Forest trained on more than 600 spectroscopically labeled sources applied to a 54,306-source photometric catalog, the observed confirmation rates are near-total for OAGBs, 95% for YSOs, 16% of CAGBs reclassified as OAGBs, only 8% for RSGs, and below 50% for PAGBs. The paper frames this as evidence that photometric surveys are a useful first pass but that spectroscopic follow-up is essential for less numerous or ov

Load-bearing premise

The result depends on the assumption that the Probabilistic Random Forest trained on about 600 spectroscopically confirmed sources, with roughly 89% accuracy, transfers cleanly to the 54,306-object photometric catalog, so the model's predictions can be treated as a spectroscopic proxy despite class imbalance and feature overlap.

Editorial extensions

If this is right

Photometric catalogs of OAGBs and YSOs in the Magellanic Clouds can be used as-is for statistical studies; their labels are largely confirmed.
Counts of RSGs and PAGBs from photometric surveys are severely contaminated; roughly 92% of photometric RSG candidates are probably other classes, mostly OAGBs.
A spectroscopically trained ML model can be applied to large photometric catalogs to flag which sources need spectroscopic follow-up, saving telescope time.
The ~16% CAGB-to-OAGB confusion means carbon-star counts from photometry should be corrected or treated as lower limits.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If the spectroscopic training sample under-represents RSGs and PAGBs, the low confirmation rates may partly reflect training-set bias rather than true photometric confusion; rebalancing the training sample would separate the two effects.
A direct test would be to obtain new spectra for a sizeable random sample of photometric RSG/PAGB candidates; the paper's 8% figure predicts almost all will be non-RSGs.
The same method could be applied to other nearby galaxies, or separately to the SMC and LMC, to see whether metallicity changes the confirmation rates.
Photometric catalogs might still be usable for rare classes if classifications are treated as probabilities and combined with priors, rather than as hard labels.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Quick read: this is a short proceedings paper with a sensible framing and a likely right qualitative conclusion, but the specific numbers in the abstract cannot be verified from what is shown. That is the main thing to know.

What is actually new: the confirmation rates for photometric labels in the Magellanic Clouds—95% YSO, 16% CAGB reclassified as OAGB, 8% RSG retained, fewer than half of PAGB—are not in the authors' earlier papers. If they hold up, they are a useful quantitative guide for planning spectroscopic follow-up. The compilation of 54,306 sources from existing catalogs and the question itself (which photometric classes can you trust) are both worthwhile. The qualitative conclusion, that photometry works for abundant classes with distinctive infrared signatures but fails for rare or overlapping classes, is almost certainly right and consistent with the prior literature.

The soft spots are real and mostly centered on the missing evidence. The model is a Probabilistic Random Forest trained on roughly 600 SAGE-Spec sources, and the only reported performance is overall ~89% accuracy. No class-wise precision, recall, or confusion matrix on the held-out spectroscopic test set is given. Because OAGBs dominate both the catalog and likely the training set, an imbalanced classifier can reach high overall accuracy while rarely predicting rare classes. So the 8% RSG confirmation rate may reflect the model's poor recall for RSGs, not photometric misclassification. That is not a minor quibble; it determines whether the paper is saying "photometric RSG labels are bad" or "the classifier almost never outputs RSG."

Second, the PAGB claim rests on 45 photometric sources. "Fewer than half confirmed" means fewer than 23 objects, with huge Poisson uncertainty, and no error bar is given. Third, Figure 1 is only a caption in the text; the confusion matrix it promises is not actually present. Fourth, the circularity concern is moderate but real: the model is trained on photometric features with spectroscopic labels and then applied to photometric labels in the same feature space. Agreement partly reflects shared photometric biases rather than independent spectral truth, so "confirmation" should be read as "model agreement" rather than "spectroscopically verified."

None of this makes the paper worthless. The framework is a reasonable first step and the qualitative takeaway is sane. But the headline percentages, which are the paper's contribution, are unsupported as presented.

Recommendation: send it to peer review, conditionally. The authors should be asked to provide the actual confusion matrix, class-wise metrics with uncertainties, prediction thresholds, and to soften the "confirmation" language. Readers who want to use these rates for survey planning should wait for the fuller version.

Referee Report

3 major / 3 minor

Summary. The paper trains a supervised machine-learning classifier (Probabilistic Random Forest) on ~600 spectroscopically labeled dusty sources in the Magellanic Clouds and applies it to a photometric catalog of 54,306 sources. It reports confirmation rates of photometric labels for five classes: OAGBs are almost all recovered, YSOs ~95%, CAGBs 16% reclassified as OAGBs, RSGs only ~8% confirmed, and PAGBs fewer than half confirmed. From this, the authors conclude that photometric classification is reliable for abundant classes with distinct infrared signatures but unreliable for rare or overlapping classes, and that spectroscopic confirmation remains necessary in the latter cases.

Significance. If the reported rates were fully supported, this would be a practical and useful quantification of photometric-classification reliability for a widely used set of Magellanic Cloud catalogs. The approach is reasonable and the qualitative conclusion is broadly consistent with expectations from class overlap and abundance. However, the central quantitative claims -- especially the small confirmation rates for RSGs and PAGBs -- are not currently backed by the evidence presented: no class-wise model confusion matrix, no precision/recall figures, no threshold description, and no discussion of transfer from the spectroscopic training sample to the full photometric catalog. The manuscript is a short proceedings contribution, but the main claim is precisely about these rates, so the missing diagnostics are load-bearing.

major comments (3)

[§3 and Figure 1] The text states that confusion matrices for each class are shown in Figure 1, but no figure content is present in the manuscript -- only the caption 'Confusion matrix comparing photometric and spectroscopic labels.' Since the confirmation rates (95%, 16%, 8%, <50%) are the paper's principal results, the actual confusion matrix, with class counts, must be included. Without it the reader cannot verify the rates or diagnose where misclassifications go.
[§2 and §3] The paper reports only an overall ~89% accuracy for the PRF model, presumably from prior work. It does not give the class-wise precision, recall, or confusion matrix for the model on held-out spectroscopic data. Because OAGBs dominate (37,950/54,306 ≈ 70%), an imbalanced model can achieve high overall accuracy while rarely predicting rare classes. The 8% RSG confirmation rate and the <50% PAGB rate could therefore reflect low model recall for those classes rather than unreliability of the photometric labels. Reporting per-class metrics for the spectroscopic test set is essential to separate these effects.
[§2 and §3] The model is trained on roughly 600 SAGE-Spec sources and applied to the full 54,306-object photometric catalog. The paper does not assess distribution shift: are the photometric colors/features of the spectroscopically confirmed sample representative of the catalogs? A simple test, such as comparing feature distributions or reporting the number of photometric sources that fall in low-density regions of the training feature space, would materially strengthen the claim that the model predictions are a valid 'spectroscopic proxy'. Without this, the confirmation rates are conditional on an unverified transfer assumption.

minor comments (3)

[§3] The statement 'fewer than half of PAGBs are confirmed' refers to a class with only 45 objects in the photometric catalog. The paper should quote the actual number (e.g., N confirmed out of 45) and, ideally, a binomial confidence interval. The rate is uninformative without the count.
[Table 1] The caption says 'after cross-matching all catalogs and removing duplicates' but the paper does not state how duplicate sources with conflicting photometric labels were handled. Since the photometric labels are the subject of the comparison, this is a potentially important source of systematic uncertainty and should be documented briefly.
[Throughout] Several references have inconsistent formatting (e.g., 'V .' for 'V.' and the arXiv identifier spacing). The reference to Ghaziasgar et al. 2025 appears twice with different author lists; please ensure the intended papers are cited unambiguously.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the ML agreement rates are an empirical comparison, not a definitional identity.

full rationale

The paper's central comparison is between photometric catalog labels and labels predicted by a supervised model trained on SAGE-Spec spectroscopic classifications. The model is trained on a separate spectroscopic sample and applied to photometric catalogs after removing cross-matched duplicates, so the prediction is not equal to the photometric label by construction. The '89% accuracy' is cited from the authors' prior work (Ghaziasgar et al. 2025b), but it is an externally testable empirical result based on held-out spectroscopic data, not an unverified assertion that the present argument reduces to. The abstract's 'confirmation rate' is strictly agreement between model predictions and photometric labels; calling this confirmation could be overstated, but it is not a circular derivation. Concerns about class imbalance, distribution shift, and missing class-wise precision/recall are validity risks for the quantitative rates, not circularity. No equation or fitted parameter is reused as a prediction, and no load-bearing conclusion is justified solely by a self-citation. Therefore no circular step is present.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

The paper rests on the validity of spectroscopic labels and the transferability of a model trained on about 600 sources to a catalog of 54,306 sources. No code, data, or hyperparameters are provided, and the quantitative confirmation rates depend on the model's behavior on the imbalanced photometric sample.

free parameters (1)

PRF hyperparameters (e.g., number of trees, depth, feature set) = unspecified
Hyperparameters are not given in this paper; the model is taken from Ghaziasgar et al. (2025b). Different hyperparameters would change the predicted labels and the confirmation rates.

assumptions (4)

domain assumption Spectroscopic labels from SAGE-Spec and archival IRS data are correct and representative of the dusty stellar populations.
The model is trained on these labels, so any errors in them propagate to the predictions. Section 2 introduces the spectroscopic dataset.
domain assumption The photometric catalogs, after cross-matching and duplicate removal, are representative of the true dusty stellar populations and contain no major contamination from unrelated sources.
The comparison treats the photometric labels as the objects under test; if the catalogs are incomplete or contaminated, the confirmation rates are biased. Section 2 describes the catalog compilation.
domain assumption The photometric features used (colors, extinction-corrected magnitudes) contain enough information to separate the five classes when combined with the PRF model.
The model's 89% accuracy on held-out spectroscopic data supports this, but the features are not listed in this paper; the transfer to the full photometric sample assumes no distribution shift.
ad hoc to paper The PRF model's 89% accuracy, as estimated in prior work, remains valid when applied to the full 54,306-object photometric catalog.
The paper applies the model without recalibration or a new validation on the photometric sample, so the reliability of model predictions is assumed to carry over. Section 3 uses these predictions as confirmations.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Comparison of Photometric and Spectroscopic Labels in Classifying Dusty Stellar Sources Using Machine Learning in the Magellanic Clouds." pith.science (2026). https://pith.science/paper/OF2ZWM5Q

@misc{pith2026250905531,
  author       = {Pith},
  title        = {Pith review of: Comparison of Photometric and Spectroscopic Labels in Classifying Dusty Stellar Sources Using Machine Learning in the Magellanic Clouds},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OF2ZWM5Q}},
  note         = {Machine review of arXiv:2509.05531}
}

read the original abstract

Dusty stellar sources, including young stellar objects (YSOs) and evolved stars such as oxygen- and carbon-rich AGBs (OAGBs, CAGBs), red supergiants (RSGs), and post-AGB stars (PAGBs), play a key role in the chemical enrichment of galaxies. Photometric surveys in the Magellanic Clouds have cataloged many such objects, but their classifications are often uncertain due to overlaps between populations. We trained machine learning models on spectroscopically labeled data from the SAGE project and applied them to photometric catalogs. The spectroscopic model achieves about 89\% accuracy. Applied to photometric labels, nearly all OAGBs are correctly identified, and YSOs have a 95\% confirmation rate. In contrast, 16\% of CAGBs are reclassified as OAGBs, only 8\% of RSGs retain their labels, and fewer than half of PAGBs are confirmed. Photometry is thus reliable for abundant populations with distinct signatures, but spectroscopic confirmation remains essential for rare or overlapping stellar classes.

Figures

Figures reproduced from arXiv: 2509.05531 by the authors.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

35 extracted references · 31 canonical work pages

[1]

T., et al

Abdollahi, H., Javadi, A., Mirtorabi, M. T., et al. 2023, ApJ, 948, 63

work page 2023
[2]

2019, arXiv:1904.07248

Baron, D. 2019, arXiv:1904.07248

arXiv 2019
[3]

L., Srinivasan, S., van Loon, J

Boyer, M. L., Srinivasan, S., van Loon, J. T., et al. 2011, AJ, 142, 103

2011
[4]

Discovery of SiC and Iron Dust Around AGB Stars in the very Metal-Poor Sextans A Dwarf Galaxy with JWST: Implications for Dust Production at High Redshift

Boyer, M. L., Sloan, G. C., Nanni, A., et al. 2025, arXiv:2507.16766

work page Pith review arXiv 2025
[5]

Cioni, M.-R. L. 2009, A&A, 496, 399

work page 2009
[6]

2024, Communications of BAO, 71, 377

Ghaziasgar, S., Abdollahi, M., Javadi, A., et al. 2024, Communications of BAO, 71, 377

work page 2024
[7]

2025, Proc

Ghaziasgar, S., Masoudnezhad, A., Javadi, A., et al. 2025, Proc. IAU Symp., 368, 67

work page 2025
[8]

2025, ApJ, 986, 168

Ghaziasgar, S., Abdollahi, M., Javadi, A., et al. 2025, ApJ, 986, 168

work page 2025

Show all 35 references

[9]

D., Meixner, M., Meade, M

Gordon, K. D., Meixner, M., Meade, M. R., et al. 2011, AJ, 142, 102

2011
[10]

A., & Chu, Y .-H

Gruendl, R. A., & Chu, Y .-H. 2009, ApJS, 184, 172 H¨ofner, S., & Olofsson, H. 2018, A&ARv, 26, 1

2009
[11]

Th., & Mirtorabi, M

Javadi, A., van Loon, J. Th., & Mirtorabi, M. T. 2011, MNRAS, 411, 263

2011
[12]

Th., Khosroshahi, H., et al

Javadi, A., van Loon, J. Th., Khosroshahi, H., et al. 2013, MNRAS, 432, 2824

2013
[13]

Th., et al

Javadi, A., Saberi, M., van Loon, J. Th., et al. 2015, MNRAS, 447, 3973

2015
[14]

C., Woods, P

Jones, O. C., Woods, P. M., Kemper, F., et al. 2017, MNRAS, 470, 3250

2017
[15]

R., & Van Winckel, H

Kamath, D., Wood, P. R., & Van Winckel, H. 2014, MNRAS, 439, 2211

2014
[16]

R., & Van Winckel, H

Kamath, D., Wood, P. R., & Van Winckel, H. 2015, MNRAS, 454, 1468

2015
[17]

M., Antoniou, V ., et al

Kemper, F., Woods, P. M., Antoniou, V ., et al. 2010, PASP, 122, 683

2010
[18]

A., Oliveira, J

Kinson, D. A., Oliveira, J. M., & van Loon, J. Th. 2021, MNRAS, 507, 5106

2021
[19]

A., Oliveira, J

Kinson, D. A., Oliveira, J. M., & van Loon, J. Th. 2022, MNRAS, 517, 140

2022
[20]

2025, arXiv:2508.05596

Mahani, H., Javadi, A., van Loon, J., et al. 2025, arXiv:2508.05596

2025 arXiv
[21]

D., Indebetouw, R., et al

Meixner, M., Gordon, K. D., Indebetouw, R., et al. 2006, AJ, 132, 2268

2006
[22]

M., van Loon, J

Pennock, C. M., van Loon, J. Th., Anih, J. O., et al. 2022, MNRAS, 515, 6046

2022
[23]

M., van Loon, J

Pennock, C. M., van Loon, J. Th., Cioni, M.-R. L., et al. 2025, MNRAS, 537, 1028 Pietrzy´nski, G., Graczyk, D., Gieren, W., et al. 2013, Nature, 495, 76

2025
[24]

2019, AJ, 157, 16

Reis, I., Baron, D., & Shahaf, S. 2019, AJ, 157, 16

2019
[25]

2010, ApJ, 723, 1195

Riebel, D., Meixner, M., Fraser, O., et al. 2010, ApJ, 723, 1195

2010
[26]

Ruffle, P. M. E., Kemper, F., Jones, O. C., et al. 2015, MNRAS, 451, 3504

2015
[27]

C., & Dopita, M

Russell, S. C., & Dopita, M. A. 1992, ApJ, 384, 508

1992
[28]

2009, AJ, 137, 4810

Srinivasan, S., et al. 2009, AJ, 137, 4810

2009
[29]

2011, ASInC, 3, 144

Subramanian, S., & Subramaniam, A. 2011, ASInC, 3, 144

2011
[30]

A., Sewilo, M., Indebetouw, R., et al

Whitney, B. A., Sewilo, M., Indebetouw, R., et al. 2008, AJ, 136, 18

2008
[31]

M., Oliveira, J

Woods, P. M., Oliveira, J. M., Kemper, F., et al. 2011, MNRAS, 411, 1597

2011
[32]

Z., Jiang, B.-W., et al

Yang, M., Bonanos, A. Z., Jiang, B.-W., et al. 2018, A&A, 616, A175

2018
[33]

Z., Jiang, B.-W., et al

Yang, M., Bonanos, A. Z., Jiang, B.-W., et al. 2019, A&A, 629, A91

2019
[34]

Z., Jiang, B.-W., et al

Yang, M., Bonanos, A. Z., Jiang, B.-W., et al. 2020, A&A, 639, A116

2020
[35]

Z., Jiang, B., et al

Yang, M., Bonanos, A. Z., Jiang, B., et al. 2021, A&A, 646, A141

2021

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.

[1] [1]

T., et al

Abdollahi, H., Javadi, A., Mirtorabi, M. T., et al. 2023, ApJ, 948, 63

work page 2023

[2] [2]

2019, arXiv:1904.07248

Baron, D. 2019, arXiv:1904.07248

arXiv 2019

[3] [3]

L., Srinivasan, S., van Loon, J

Boyer, M. L., Srinivasan, S., van Loon, J. T., et al. 2011, AJ, 142, 103

2011

[4] [4]

Discovery of SiC and Iron Dust Around AGB Stars in the very Metal-Poor Sextans A Dwarf Galaxy with JWST: Implications for Dust Production at High Redshift

Boyer, M. L., Sloan, G. C., Nanni, A., et al. 2025, arXiv:2507.16766

work page Pith review arXiv 2025

[5] [5]

Cioni, M.-R. L. 2009, A&A, 496, 399

work page 2009

[6] [6]

2024, Communications of BAO, 71, 377

Ghaziasgar, S., Abdollahi, M., Javadi, A., et al. 2024, Communications of BAO, 71, 377

work page 2024

[7] [7]

2025, Proc

Ghaziasgar, S., Masoudnezhad, A., Javadi, A., et al. 2025, Proc. IAU Symp., 368, 67

work page 2025

[8] [8]

2025, ApJ, 986, 168

Ghaziasgar, S., Abdollahi, M., Javadi, A., et al. 2025, ApJ, 986, 168

work page 2025