REVIEW 4 major objections 7 minor 53 references
Capturing star formation activity from compressed photometric images of galaxies
T0 review · 4 major / 7 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read A Vision Transformer trained on 124,000 compressed survey color images classifies star-forming galaxies with F1=0.85 and predicts emission-line ratios with R^2≈0.83, suggesting spectroscopy can be bypassed for the star-formation question.
desk verdict A clean, honest image-only star-formation classifier that is missing the color-only control needed to support its central claim about visual features. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is the Vision Transformer base model with 16×16 patches on 224×224 images: each image is split into patches, embedded into a 768-dimensional space, and summarized by a [CLS] token through 12 self-attention blocks, so local clumpy star-forming regions and global spiral structure both enter the prediction. Pre-training on natural images is transferred to galaxy images, and task-specific heads turn the [CLS] representation into either two class logits or two continuous line-ratio outputs. Self-attention over patches is what lets the model combine color and spatial pattern without hand-designed morphology features.
What would settle it
Take the same train/test split and fit a classifier using only the $g-r$ color (or any single color index) of each galaxy image; if a color-only baseline reaches F1≈0.85 and $R^2\approx0.83$, the claim that visual morphology carries the star-forming signal is falsified.
Extended reading notes
Core claim
On its own terms, the paper's discovery is that a galaxy's spectroscopic ionization state leaves a recoverable imprint in its optical appearance. A Vision Transformer base model, fine-tuned on the g, r, i color composites, classifies star-forming versus non-star-forming galaxies with precision 0.85, recall 0.86, and F1 0.85; for the galaxies it labels star-forming, it predicts $\log([{\rm N\,II}]λ6583/{\rm H}\alpha)$ and $\log([{\rm O\,III}]λ5007/{\rm H}\beta)$ with $R^2 = 0.837$ and $0.830$, tracing the star-forming ridge of the BPT diagram with reduced scatter. The authors also report that the mapping does not transfer to $z\approx0.16$-$0.17$ and that nuclear-dominated AGN ratios are not recoverable, because the nuclear emission occupies less than a pixel in the compressed images.
Load-bearing premise
The predictive signal must come from spatial or visual features, not just from the integrated color information already encoded in the JPEG; the paper tests no simple $g-r$ color baseline, so if such a cut matches F1≈0.85, the direct-link claim collapses to a color statement.
Editorial extensions
If this is right
- If the claim holds, star-forming galaxy catalogs for future wide surveys can be produced from survey images alone, turning a spectroscopy-limited question into an imaging pipeline.
- The regression result implies that BPT-style line ratios of star-forming galaxies can be ranked or binned from images, enabling statistical studies of ionization and metallicity without individual spectra.
- The model's sharp drop at $z\approx0.16$-$0.17$ defines the usable redshift range: the visual-to-spectroscopic mapping works only where galaxy structure is resolved in more than a few pixels.
- The reported accuracy applies only to the coarse two-way split; the four-way BPT classes (star-forming, composite, Seyfert, LINER) are not recovered from JPEGs.
- Because false negatives are biased toward massive star-forming galaxies, downstream science using this classifier must account for a mass-dependent incompleteness.
Reading between the lines
- A color-only shortcut is the most serious untested alternative: a simple $g-r$ (or similar color-index) threshold could plausibly reproduce much of the F1, since star-forming galaxies are predominantly blue and JPEG composites encode color; the paper provides no such baseline, so its visual-features interpretation remains open.
- A direct way to isolate what the network actually uses is to feed grayscale versions of the same images or to mask the central pixel region; if performance persists, the model relies on light distribution and morphology rather than on color alone.
- The same architecture could act as an anomaly detector: galaxies whose predicted emission-line ratios disagree strongly with measured ones are likely interacting, edge-on, or hosting unresolved nuclear activity, exactly the cases the paper shows are misclassified.
- The higher-redshift failure mode points to resolution rather than intrinsic physics, so deep, high-resolution imaging from upcoming surveys may recover the mapping if angular size is the limiting factor.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper trains a Vision Transformer (ViT) on approximately 124,000 SDSS gri color-composite JPEG images of galaxies at 0.01<z<0.06 to perform two tasks: (1) binary classification of galaxies into star-forming versus non-star-forming (with non-star-forming defined as the union of composite, Seyfert, LINER, weak-emission, and non-emission galaxies), and (2) regression of the [NII]λ6583/Hα and [OIII]λ5007/Hβ line ratios. The spectroscopic labels are taken from the OSSY catalog. The authors report a test-set F1 score of 0.85 for classification and R² values of 0.837 and 0.830 for the two line-ratio regressions. They further evaluate the model on unseen redshift bins (0.06<z<0.07 and 0.16<z<0.17), finding comparable performance at the lower bin and a marked degradation (F1=0.56) at the higher bin. The paper interprets these results as evidence that photometric JPEG images alone can directly capture star-forming activity, bypassing traditional spectroscopic diagnostics.
Significance. If the central claim is robust, the method would provide a cheap, fast screening tool for star-forming galaxies in large photometric surveys, which is a practically useful contribution. The paper is honest about limitations: it reports the high-redshift degradation, the failure for Seyferts and LINERs, and the lossy nature of JPEG compression. It also provides code and data through a public repository, which is commendable. The external OSSY labels make the supervised mapping non-circular. However, the significance of the claimed 'direct link' between visual features and spectroscopic state is currently underevidenced, because the paper never rules out that the model is simply exploiting the well-known color–BPT correlation (e.g., blue star-forming versus red passive galaxies). The novelty and interpretation therefore rest on a missing control experiment.
major comments (4)
- [Section 4.1, Section 6] The central claim of a 'direct link between apparent visual features and underlying spectroscopic characteristics' (Section 6) requires demonstrating that the ViT captures spatial/morphological information beyond integrated color. The paper never compares against a simple color-based baseline, such as a g-r color cut or a logistic regression on SDSS g-r, u-r, and r-i colors (or on the average RGB of the same JPEG images). Since star-forming galaxies are systematically bluer (the paper itself cites Strateva et al. 2001), a color-only model could plausibly match F1=0.85, especially given the coarse binary grouping that lumps all non-star-forming types together. Please add such baselines and report F1, precision, recall, and regression R² for them. If the ViT does not significantly outperform the color-only model, the abstract's claim that images 'directly' encode star-forming activity collapses to a restatement of known color–BPT correlations.
- [Section 4.2, Figure 5] The regression quality is only reported via R², RMSE, and MAE, but the authors themselves state in Section 4.2 that predictions are 'skewed toward the peak values' and that the predicted BPT diagram shows 'smaller dispersion' than the ground truth. These statements indicate strong shrinkage toward the mean, which can inflate R² when the test labels are concentrated near the mode. Please quote the linear-fit slopes and intercepts (currently only in the figure) and compare the RMSE (0.089 and 0.161) with the standard deviation or 16–84th percentile width of the ground-truth line-ratio distributions. This is needed to assess whether the regression predicts anything beyond the median of the training population.
- [Section 4.3, Section 6] The abstract and Section 6 state that the method is promising for Euclid, DES, and LSST, but the extrapolation to 0.16<z<0.17 yields F1=0.56, which the authors attribute to reduced angular size and loss of resolved features. Since LSST and Euclid will predominantly deliver galaxies at z>0.1, the claimed applicability is not supported by the reported experiments. Either the survey-applicability statements should be substantially tempered, or the authors should provide evidence of a transferable variant (e.g., training on higher-redshift galaxies, using larger cutouts, or leveraging multi-band coadds with better resolution). As written, the high-redshift degradation undermines the stated practical motivation.
- [Section 2, Appendix C] The binary classification collapses composites, Seyferts, LINERs, weak-emission, and non-emission galaxies into a single 'non-star-forming' class. The confusion matrix in Figure 4 does not break down performance by subtype, yet composites are known to be intermediate in color and often classified as star-forming by color-based methods. Please provide a subtype-level confusion matrix (e.g., precision/recall for star-forming versus composite versus passive galaxies) to clarify whether the model is separating 'star-forming' from 'everything else' by color or by actual star-formation indicators. This is directly relevant to the baseline concern raised above.
minor comments (7)
- [Section 4.2] Please state explicitly that all emission-line ratios are logarithmic (log10) before reporting numerical values such as RMSE=0.089; the axes of Figure 5 are not labeled in the text, and the numerical errors are ambiguous without this unit convention.
- [Section 1] The first paragraph contains a typo: 'Sloand Digital Sky Survey' should be 'Sloan Digital Sky Survey'.
- [Section 5] The heading 'LIMITATIONS AND CA VEATS' should read 'LIMITATIONS AND CAVEATS'.
- [Section 4.3] The phrase 'c.f.,' should be 'cf.,' for the Latin abbreviation, and the sentence 'when the extract features are insufficient' in Section 4.2 should read 'when the extracted features are insufficient'.
- [Section 3.2.3] The authors state that the best model is selected based on RMSE (regression) or accuracy (classification) on checkpoints saved every 100 steps, but they do not report the final epoch or step at which the best model was obtained; please provide this information for reproducibility.
- [Section 2] The text says a lower redshift limit is imposed but does not specify it in Section 2; the abstract states z=0.01, so please state the lower limit explicitly in the data section.
- [Section B, Figure 9] The discussion of morphology notes that 'uncertain' objects dominate all groups (>50%) and have small apparent sizes, but the text does not explicitly caution that the Galaxy Zoo morphological interpretation is therefore limited to a minority of relatively large, clearly classified objects; please add such a caveat.
Circularity Check
No significant circularity: the ViT is trained on external spectroscopic labels and evaluated on held-out data, so the results do not reduce to the paper's own inputs.
full rationale
The paper's derivation chain is a standard supervised learning pipeline: it takes SDSS gri composite JPEG images as inputs and trains a Vision Transformer against spectroscopic labels and line ratios from the OSSY catalog (Oh et al. 2011) and the follow-up measurements of Oh et al. (2015). The labels are generated externally, using Gaussian amplitude-to-noise thresholds and BPT demarcation lines from Kewley et al. (2001), Kauffmann et al. (2003b), Kewley et al. (2006), and Schawinski et al. (2007), not from the image model itself. The reported F1 and R^2 values are evaluated on a held-out 20% test split (Section 3.2.3) and on disjoint redshift bins (Section 4.3), so the predictions are not refits of parameters derived from the same data. The only self-referential element is that the first author co-authored the OSSY catalog, but that catalog is used as an external ground-truth resource with published line measurements and does not presuppose the visual-features-to-spectroscopy claim. The absence of a color-only baseline is a legitimate scientific weakness about what information the model uses, but it is not a circularity: no equation in the paper defines the image representation in terms of the BPT classes or line ratios, and no fitted quantity is renamed as a prediction. Thus no circular step can be exhibited, and the appropriate score is 0.
Assumptions & free parameters
free parameters (1)
- A/N>=3 threshold for emission-line classification =
3
assumptions (4)
- domain assumption OSSY catalog emission-line measurements are accurate enough to serve as ground truth.
- domain assumption JPEG color composite images retain sufficient information to infer BPT line ratios.
- ad hoc to paper The SDSS gri color composite mapping preserves the physical color information correlated with BPT activity.
- domain assumption The random 80/20 split does not create train-test overlap that inflates performance.
Cite this review
Pith. "Pith review of Capturing star formation activity from compressed photometric images of galaxies." pith.science (2026). https://pith.science/paper/HU4XCJGN
@misc{pith2026250723019,
author = {Pith},
title = {Pith review of: Capturing star formation activity from compressed photometric images of galaxies},
year = {2026},
howpublished = {\url{https://pith.science/paper/HU4XCJGN}},
note = {Machine review of arXiv:2507.23019}
}
abstract
We present a novel approach for classifying star-forming galaxies using photometric images. By utilizing approximately $124,000$ optical color composite images and spectroscopic data of nearby galaxies at $0.01<z<0.06$ from the Sloan Digital Sky Survey, along with follow-up spectroscopic line measurements from the OSSY catalog, and leveraging the Vision Transformer machine-learning technique, we demonstrate that galaxy images in JPEG format alone can be directly used to determine whether star-forming activity dominates the galaxy, bypassing traditional spectroscopic analyses such as emission-line diagnostic diagrams. We anticipate that this method holds significant potential for application in current and future large-scale surveys, such as Euclid, the Dark Energy Survey (DES), and the Legacy Survey of Space and Time (LSST).
Figures
Figures from the paper (8 more)
Reference graph
Works this paper leans on
-
[1]
, " * write output.state after.block = add.period write newline
ENTRY address archivePrefix author booktitle chapter doi edition editor eprint howpublished institution journal key month number organization pages publisher school series title misctitle type volume year version url label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts ...
-
[2]
write newline
" write newline "" before.all 'output.state := FUNCTION format.url url empty "" new.block "" url * "" * if FUNCTION format.eprint eprint empty "" archivePrefix empty "" archivePrefix "arXiv" = new.block " " eprint * " " * new.block " " eprint * " " * if if if FUNCTION format.doi doi empty "" " " doi * " " * if FUNCTION format.pid doi empty eprint empty ur...
-
[3]
HD d d d d d d d d d d d d d d uJN1 2 H d d d d d d d d d d d d d d d k ?, Ʋ ۋk
thebibliography [1] 20pt to REFERENCES 6pt =0pt -12pt 10pt plus 3pt =0pt =0pt =1pt plus 1pt =0pt =0pt -12pt =13pt plus 1pt =20pt =13pt plus 1pt \@M =10000 =-1.0em =0pt =0pt 0pt =0pt =1.0em @enumiv\@empty 10000 10000 `\.\@m \@noitemerr \@latex@warning Empty `thebibliography' environment \@ifnextchar \@reference \@latexerr Missing key on reference command E...
2021
-
[4]
Abazajian , K. N., Adelman-McCarthy , J. K., Ag \"u eros , M. A., et al. 2009, , 182, 543, 10.1088/0067-0049/182/2/543
-
[5]
Abbott , T. M. C., Abdalla , F. B., Allam , S., et al. 2018, , 239, 18, 10.3847/1538-4365/aae9f0
-
[6]
K., Glazebrook , K., Brinkmann , J., et al
Baldry , I. K., Glazebrook , K., Brinkmann , J., et al. 2004, , 600, 681, 10.1086/380092
doi:10.1086/380092 2004
-
[7]
Baldwin , J. A., Phillips , M. M., & Terlevich , R. 1981, , 93, 5, 10.1086/130766
doi:10.1086/130766 1981
-
[8]
Balogh , M. L., Morris , S. L., Yee , H. K. C., Carlberg , R. G., & Ellingson , E. 1999, , 527, 54, 10.1086/308056
doi:10.1086/308056 1999
Show all 53 references
-
[9]
F., McIntosh , D
Bell , E. F., McIntosh , D. H., Katz , N., & Weinberg , M. D. 2003, , 149, 289, 10.1086/378847
2003 doi
-
[10]
R., & Moustakas , J
Blanton , M. R., & Moustakas , J. 2009, , 47, 159, 10.1146/annurev-astro-082708-101734
2009 doi
-
[12]
2024, , 683, A42, 10.1051/0004-6361/202348544
Cao , J., Xu , T., Deng , Y., et al. 2024, , 683, A42, 10.1051/0004-6361/202348544
2024 doi
-
[13]
J., Arag \'o n-Salamanca , A., et al
Cheng , T.-Y., Conselice , C. J., Arag \'o n-Salamanca , A., et al. 2020, , 493, 4209, 10.1093/mnras/staa501
2020 doi
-
[14]
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. 2019, in Proceedings of the 2019 Conference of the North A merican Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), ed. J. Burstein, C. Doran, & T. Solori...
2019 doi
-
[15]
W., & Dambre , J
Dieleman , S., Willett , K. W., & Dambre , J. 2015, , 450, 1441, 10.1093/mnras/stv632
2015 doi
-
[16]
Dom \' nguez S \'a nchez , H., Huertas-Company , M., Bernardi , M., Tuccillo , D., & Fischer , J. L. 2018, , 476, 3661, 10.1093/mnras/sty338
2018 doi
-
[17]
2021, in International Conference on Learning Representations
Dosovitskiy, A., Beyer, L., Kolesnikov, A., et al. 2021, in International Conference on Learning Representations. https://openreview.net/forum?id=YicbFdNTTy
2021
-
[18]
M., Iglesias-P \'a ramo , J., et al
Duarte Puertas , S., Vilchez , J. M., Iglesias-P \'a ramo , J., et al. 2017, , 599, A71, 10.1051/0004-6361/201629044
2017 doi
-
[19]
2024, arXiv e-prints, arXiv:2405.13491, 10.48550/arXiv.2405.13491
Euclid Collaboration , Mellier , Y., Abdurro'uf , et al. 2024, arXiv e-prints, arXiv:2405.13491, 10.48550/arXiv.2405.13491
2024 doi
-
[20]
2021, arXiv preprint arXiv:2107.01682
Gao, X., Qian, Y., & Gao, A. 2021, arXiv preprint arXiv:2107.01682
2021 arXiv
-
[21]
2022, in 2022 44th Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), IEEE, 480--483
Gheflati, B., & Rivaz, H. 2022, in 2022 44th Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), IEEE, 480--483
2022
-
[22]
K., & Desai , S
Gupta , R., Srijith , P. K., & Desai , S. 2022, Astronomy and Computing, 38, 100543, 10.1016/j.ascom.2021.100543
2022
-
[23]
Heckman , T. M. 1980, , 87, 152
1980
-
[24]
M., Tyson , J
Ivezi \'c , Z ., Kahn , S. M., Tyson , J. A., et al. 2019, , 873, 111, 10.3847/1538-4357/ab042c
2019 doi
-
[25]
2022, in Mach
Kang, S., Shin, M.-S., & Kim, T. 2022, in Mach. Learn. Phys. Sci. workshop
2022
-
[26]
M., White , S
Kauffmann , G., Heckman , T. M., White , S. D. M., et al. 2003 a , , 341, 33, 10.1046/j.1365-8711.2003.06291.x
2003
-
[27]
M., Tremonti , C., et al
Kauffmann , G., Heckman , T. M., Tremonti , C., et al. 2003 b , , 346, 1055, 10.1111/j.1365-2966.2003.07154.x
2003
-
[28]
Kaviraj , S., Schawinski , K., Devriendt , J. E. G., et al. 2007, , 173, 619, 10.1086/516633
2007 doi
-
[29]
J., Dopita , M
Kewley , L. J., Dopita , M. A., Sutherland , R. S., Heisler , C. A., & Trevena , J. 2001, , 556, 121, 10.1086/321545
2001 doi
-
[31]
2021, , 656, A140, 10.1051/0004-6361/202140757
Kumari , N., Maiolino , R., Trussler , J., et al. 2021, , 656, A140, 10.1051/0004-6361/202140757
2021 doi
-
[32]
2021, , 162, 297, 10.3847/1538-3881/ac2e96
Lee , J., & Shin , M.-S. 2021, , 162, 297, 10.3847/1538-3881/ac2e96
2021 doi
-
[33]
Y.-Y., Liao, S.-M., Huang, H.-J., Kuo, W.-T., & Ou, O
Lin, J. Y.-Y., Liao, S.-M., Huang, H.-J., Kuo, W.-T., & Ou, O. H.-M. 2021, arXiv preprint arXiv:2110.01024
2021 arXiv
-
[34]
2011, , 410, 166, 10.1111/j.1365-2966.2010.17432.x
Lintott , C., Schawinski , K., Bamford , S., et al. 2011, , 410, 166, 10.1111/j.1365-2966.2010.17432.x
2011
-
[35]
2018, Mixed Precision Training
Micikevicius, P., Narang, S., Alben, J., et al. 2018, Mixed Precision Training. 1710.03740
2018 arXiv
-
[36]
Oh , K., Sarzi , M., Schawinski , K., & Yi , S. K. 2011, , 195, 13, 10.1088/0067-0049/195/2/13
2011 doi
-
[37]
K., Schawinski , K., et al
Oh , K., Yi , S. K., Schawinski , K., et al. 2015, , 219, 1, 10.1088/0067-0049/219/1/1
2015 doi
-
[38]
E., & Ferland , G
Osterbrock , D. E., & Ferland , G. J. 2006, Astrophysics of gaseous nebulae and active galactic nuclei
2006
-
[39]
2024, , 531, 4990, 10.1093/mnras/stae1450
Parker , L., Lanusse , F., Golkar , S., et al. 2024, , 531, 4990, 10.1093/mnras/stae1450
2024 doi
-
[40]
F., Jim \'e nez-Vicente , J., et al
S \'a nchez-Bl \'a zquez , P., Peletier , R. F., Jim \'e nez-Vicente , J., et al. 2006, , 371, 703, 10.1111/j.1365-2966.2006.10699.x
2006
-
[41]
2007, , 382, 1415, 10.1111/j.1365-2966.2007.12487.x
Schawinski , K., Thomas , D., Sarzi , M., et al. 2007, , 382, 1415, 10.1111/j.1365-2966.2007.12487.x
2007
-
[42]
D., & Zhang , C
Schawinski , K., Turp , M. D., & Zhang , C. 2018, , 616, L16, 10.1051/0004-6361/201833800
2018 doi
-
[43]
Schawinski , K., Zhang , C., Zhang , H., Fowler , L., & Santhanam , G. K. 2017, , 467, L110, 10.1093/mnrasl/slx008
2017 doi
-
[44]
2023, , 135, 084101, 10.1088/1538-3873/ace851
Seo , E., Kim , S., Lee , Y., et al. 2023, , 135, 084101, 10.1088/1538-3873/ace851
2023 doi
-
[45]
2018, , 477, 2513, 10.1093/mnras/sty764
Stark , D., Launet , B., Schawinski , K., et al. 2018, , 477, 2513, 10.1093/mnras/sty764
2018 doi
-
[46]
R., et al
Strateva , I., Ivezi \'c , Z ., Knapp , G. R., et al. 2001, , 122, 1861, 10.1086/323301
2001 doi
-
[47]
A., Weinberg , D
Strauss , M. A., Weinberg , D. H., Lupton , R. H., et al. 2002, , 124, 1810, 10.1086/342343
2002 doi
-
[48]
2022, Injury, 53, 2625
Tanzi, L., Audisio, A., Cirrincione, G., Aprato, A., & Vezzetti, E. 2022, Injury, 53, 2625
2022
-
[49]
Tarsitano , F., Bruderer , C., Schawinski , K., & Hartley , W. G. 2022, , 511, 3330, 10.1093/mnras/stac233
2022 doi
-
[50]
C., Faber , S
Trager , S. C., Faber , S. M., Worthey , G., & Gonz \'a lez , J. J. 2000, , 119, 1645, 10.1086/301299
2000 doi
-
[51]
2017, in Advances in Neural Information Processing Systems, 5998--6008
Vaswani, A., Shazeer, N., Parmar, N., et al. 2017, in Advances in Neural Information Processing Systems, 5998--6008
2017
-
[52]
Veilleux , S., & Osterbrock , D. E. 1987, , 63, 295, 10.1086/191166
1987 doi
-
[53]
2020, , 491, 1554, 10.1093/mnras/stz2816
Walmsley , M., Smith , L., Lintott , C., et al. 2020, , 491, 1554, 10.1093/mnras/stz2816
2020 doi
-
[54]
M., Gonzalez , J
Worthey , G., Faber , S. M., Gonzalez , J. J., & Burstein , D. 1994, , 94, 687, 10.1086/192087
1994 doi
-
[55]
G., Adelman , J., Anderson , Jr., J
York , D. G., Adelman , J., Anderson , Jr., J. E., et al. 2000, , 120, 1579, 10.1086/301513
2000 doi
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.