REVIEW 4 major objections 6 minor 25 references
Galaxy Morphological Classification with Manifold Learning
T0 review · 4 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash
Pith's one-line read On Galaxy Zoo DECaLS images, Locally Linear Embedding followed by a logistic regression classifier reaches accuracies comparable to simple neural networks, and the three-dimensional embedding remains physically interpretable.
desk verdict Useful benchmark and a genuinely interesting interpretability result, but the smooth/featured numbers contradict each other and need to be reconciled before the main claim is credible. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the Locally Linear Embedding itself: LLE expresses every high-dimensional image as a linear combination of its neighbors and preserves those local reconstruction weights in a low-dimensional space, so nearby images stay nearby. The paper's three-dimensional version of this embedding supplies the interpretable coordinate system, and the interpretation is carried by a closed-form parameterization of elliptical galaxy images that maps the semimajor axis $a$, semiminor axis $b$, and orientation $\varphi$ to embedded coordinates using the third flattening measure $n=(a-b)/(a+b)$ times $\cos 2\varphi$ and $\sin 2\varphi$. This map predicts the almond shape of the manifold, explains the spinoric half-rotation, and lets the authors check the real-data embedding against synthetic ellipses.
What would settle it
Evaluate the same LLE-plus-logistic-regression pipeline on Galaxy Zoo DECaLS galaxies whose vote fractions fall between 0.3 and 0.7, the region the current curation discards, and compare accuracy on that ambiguous set with the reported 95 percent. In parallel, measure whether the embedding's azimuthal coordinate tracks the independently computed third flattening measure on synthetic ellipses with known axes. If accuracy collapses on the ambiguous set, or the azimuthal correlation is absent on the synthetic set, the paper's central claims are falsified.
Extended reading notes
Core claim
The central claim is that Locally Linear Embedding is the dimensionality-reduction method of choice for classical classifiers on galaxy morphology: across the classifiers tested, LLE consistently yields the highest subsequent accuracy, and the margin is achieved with only weak sensitivity to hyperparameters. On a 20,000-image subsample, LLE followed by logistic regression reaches 0.95 weighted F1 for round/in-between/cigar, 0.85 for smooth/featured, and 0.96 for edge-on/face-on, matching a three-layer perceptron and coming within a few points of a small convolutional network. The paper further claims that the three-dimensional LLE embedding of the shape classes is genuinely interpretable rather than a black box: it forms an almond-shaped manifold whose conical coordinates correspond to the galaxy's third flattening measure, orientation angle, and total intensity or scale, and a full revolution of the polar angle induces a half-rotation of the galaxy image. The same structure appears in a synthetic dataset of stretched and rotated ellipses, fitted by the closed-form map $(a,b,\varphi) \mapsto (k_1 ab (a-b)/(a+b)\cos 2\varphi,\ k_1 ab (a-b)/(a+b)\sin 2\varphi,\ k_2 ab)$.
Load-bearing premise
The comparison depends on Galaxy Zoo volunteer vote fractions, cut at class fraction above 0.8 (or below 0.2 for featured) with more than ten voters, being accurate ground truth for morphology; if those thresholded labels are biased or unrepresentative, the reported accuracies will not generalize to the full survey.
Editorial extensions
If this is right
- If LLE is the best front end for classical classifiers, survey-scale morphology pipelines need no deep feature extractor to reach competitive accuracy.
- The three interpretable coordinates give a direct handle on physical parameters - flattening, orientation, and scale - without fitting explicit models to each image.
- The weak dependence on hyperparameters means the pipeline can be deployed without expensive per-dataset tuning searches.
- The absence of a clear cluster structure, except a Davies-Bouldin preference for four clusters, suggests morphology is better represented as a continuum than as separated groups in unsupervised settings.
- The modest computational cost makes the approach feasible for large upcoming surveys and for fast unsupervised classification without heavy pretrained models.
Reading between the lines
- The almond manifold's two angular coordinates effectively parameterize centered ellipses up to rotation and scale, so the embedding could be reused as a source of rotation- and scale-normalized features for other classifiers.
- The $2\varphi$ factors mean the embedding automatically encodes 180-degree rotational symmetry; this might replace explicit rotation augmentation in shape classifiers.
- Because preprocessing crops by Petrosian radius, the z-coordinate labeled as scale may partly encode the crop-window normalization rather than physical galaxy size; separating these would clarify how much interpretability survives outside this dataset.
- The 0.8 vote-fraction threshold excludes genuinely ambiguous galaxies, so testing the pipeline on the full vote distribution would show whether the reported accuracy extends to the hardest cases.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript presents a systematic comparison of five dimensionality-reduction methods (LLE, Isomap, UMAP, t-SNE, PCA) combined with classical classifiers (logistic regression, SVC, decision tree) for three Galaxy Zoo DECaLS morphology classification tasks. It reports that LLE yields the best downstream accuracy, that LLE plus logistic regression attains accuracy comparable to simple neural networks, and that a 3D LLE/Isomap/PCA embedding of the round/in-between/cigar data is interpretable in terms of ellipse parameters. It also applies LLE followed by k-means clustering and evaluates cluster validity indices. The paper includes public code and interactive demos.
Significance. If the reported performance and interpretability are correct, the paper provides a useful, computationally light, and interpretable alternative to deep networks for survey-scale morphology classification, especially for fast exploratory analysis. The strengths are the breadth of the method comparison, the public code repository, and the synthetic-model construction made available for testing. However, the internal numerical inconsistencies in the main classification tables and the unclear neural-network comparison protocol prevent the central claim from being accepted as stated.
major comments (4)
- [Section 3.2, Figure 8, Table 5] The smooth/featured confusion matrix in Figure 8 is inconsistent with the classification report in Table 5. Figure 8 gives 2428 correct smooth and 2267 correct featured out of 6000 test images, i.e., an accuracy of 0.7825, whereas Table 5 reports accuracy 0.85. In addition, the row sums of Figure 8 are 3081 for smooth and 2919 for featured, while Table 5 lists support 3103 and 2897, and the per-class precision/recall values (0.86/0.85 and 0.85/0.85) cannot be recovered from the matrix. This contradiction directly affects the conclusion that LLE plus logistic regression is comparable to the FCNN (Table 11, accuracy 0.83) for texture classification. The authors must recompute the classification report and correct either the figure or the table before the central claim can be assessed.
- [Section 3.2 vs Appendix Tables 13-14] The main text states that the smooth/featured and edge-on/face-on experiments 'use the same parameters as in the previous case' (LLE components 138, neighbors 10, from Table 3), yet Appendix Tables 13 and 14 report fine-tuned parameters for exactly these two tasks, namely LLE 64 components with 96 neighbors and LLE 199 components with 199 neighbors. The manuscript needs to state unambiguously which parameter sets produced Figures 8 and 10 and Tables 5 and 6. If the fine-tuned parameters were used, the sentence about using the same parameters is wrong; if they were not used, the Appendix tables are mislabeled.
- [Section 2.4 and Appendix Tables 7-12] The neural-network classification reports in Tables 7-12 list support 20,000, whereas the LLE plus logistic regression reports in Tables 4-6 list support 6,000. According to Section 2.4, the neural networks are trained on 14,000 images and tested on 6,000, so a test-set report should have support 6,000. The reported 20,000-support numbers appear to refer to the 'different random sample of 20,000 images' used for additional control. As a result, the comparison in Section 3.2 between LLE plus logistic regression and the neural networks may not be on the same held-out test set. Please report the neural-network metrics on the same 6,000-image test set as the LLE pipeline, or demonstrate that the 20,000-sample evaluation is comparable.
- [Section 4, Eq. (1)] The interpretability claim is not quantified. The assertion that in the 3D embedding the azimuthal angle correlates with the third flattening measure, the polar angle with the orientation angle, and the z-component with total intensity is supported mainly by a synthetic model with a hand-chosen profile exponent (-2.5) and a mapping with free scaling constants k1 and k2. The paper should provide quantitative evidence on the real data, e.g., Spearman or Pearson correlation coefficients between the embedding coordinates (after the conical-coordinate transformation) and the ellipse parameters, or the residual of the fitted mapping. Without such numbers, 'correlates' is a qualitative visual statement and the abstract's interpretability claim is not fully established.
minor comments (6)
- [Section 2.1, Table 1] The sentence 'a single image can belong to several classes if they come from different classification problems, thus the total number of images is just 55137' is confusing; it should say that the curated dataset contains 55,137 unique images because the class-specific subsets overlap across the three classification schemes. Also clarify whether the 'unsupervised' subset is the same as the full curated set.
- [Section 2.3] The parameter optimization is performed using 5-fold cross-validation on a sample of 5,000 images, while the final evaluation uses a 14,000/6,000 split on 20,000 images. The text should state explicitly whether the optimized parameters were re-selected or fixed for the final run, and whether the reported accuracy is on the held-out test fold.
- [Section 2.3 and Table 2] The note that 'the number of components for t-SNE should be greater than three times the perplexity' is inconsistent with Table 2, which fixes components at 2 while allowing perplexity up to 50. Please clarify the constraint or correct the table.
- [Section 3.2] The phrase 'the multilayer perceptron (fully-connected neural network, FCNN) exhibits the same performance' is ambiguous; it should specify that the FCNN performance is similar to that of the LLE plus logistic regression pipeline, not to the CNN.
- [Section 4] The term 'spinoric property' is introduced without definition; explain what it means for the 3D representation and how the half-rotation of the galaxy image was verified.
- [Section 3.3] The claim that four clusters 'closely align' with elliptical, spiral, lenticular, and irregular Hubble types is not directly supported, because the classification tasks in this paper use the round/in-between/cigar, smooth/featured, and edge-on/face-on schemes rather than the four Hubble bins. Please temper or justify this interpretation.
Circularity Check
Classification benchmarks are externally anchored and not circular; the interpretability claim is a post-hoc model-based construction, not a derived prediction.
-
other
[Section 4, Interpretability (formula after 'we have found an approximation...')]
"Moreover, the extracted dimensions have an interpretation ... the azimuthal angle θ correlates with the third flattening measure ..., the polar angle φ with the galaxy's orientation angle, and the z-component with the total intensity or scale ... We have found an approximation that can be expressed ... as (a,b,φ) ↦ (k1 ab (a−b)/(a+b) cos 2φ, k1 ab (a−b)/(a+b) sin 2φ, k2 ab)."
The claimed physical meaning is imported from the synthetic model's generating formula: the coordinate directions are defined as functions of n, φ, and ab, so in the synthetic dataset the 'correlations' are true by construction. The real-data interpretability conclusion is then supported only by visual similarity to this self-constructed manifold, with free scaling parameters k1 and k2 and a hand-chosen profile, rather than by an independent measurement of those correlations. This is a post-hoc explanatory fit, not a prediction derived from LLE. It does not affect the supervised classification results, which are validated against external Galaxy Zoo labels.
full rationale
The central classification claim (LLE + logistic regression achieving accuracy comparable to simple neural networks) is not circular: the labels come from external Galaxy Zoo volunteer votes, hyperparameters are optimized on one sample and evaluated on a separate 20,000-image test split, and the neural-network comparison is an independently trained benchmark. No load-bearing self-citation appears; the only author citations are to public code and demo repositories. The one partially self-confirming element is the interpretability section, where the synthetic model's coordinates encode the same ellipse parameters that are later said to 'interpret' the embedding; this is a post-hoc modeling analogy rather than a forced derivation. Separately, the paper contains a non-circular but checkable internal inconsistency: the smooth/featured accuracy in Table 5 (0.85) and its per-class precision/recall cannot be reconciled with the confusion matrix in Figure 8, which implies (2,428 + 2,267)/6,000 ≈ 0.78. This matters for the 'comparable to FCNN (0.83)' claim on that task, but it is a numerical/correctness issue, not circularity. The threshold-based Galaxy Zoo labels are a validity and generalization concern, not a circularity concern.
Assumptions & free parameters
free parameters (4)
- LLE embedding dimension and neighbors =
138 components, 10 neighbors (main result)
- k1, k2 scaling constants in interpretability mapping =
not reported
- Synthetic model profile exponent =
-2.5
- Image crop size =
120x120
assumptions (4)
- domain assumption Galaxy Zoo volunteer vote fractions with thresholds (class fraction >0.8, >10 voters) define reliable ground-truth labels.
- domain assumption The 120x120 central grayscale crop preserves the visual information needed for morphology classification.
- domain assumption The scikit-learn and RAPIDS cuml implementations of LLE, Isomap, UMAP, t-SNE, and PCA are numerically correct.
- domain assumption Galaxy images lie on a low-dimensional manifold parameterized approximately by semimajor axis, semiminor axis, and orientation angle.
Cite this review
Pith. "Pith review of Galaxy Morphological Classification with Manifold Learning." pith.science (2026). https://pith.science/paper/OTKQFPXN
@misc{pith2026241209358,
author = {Pith},
title = {Pith review of: Galaxy Morphological Classification with Manifold Learning},
year = {2026},
howpublished = {\url{https://pith.science/paper/OTKQFPXN}},
note = {Machine review of arXiv:2412.09358}
}
read the original abstract
We address the problem of morphological classification of galaxies from the Galaxy Zoo DECaLS dataset using classical machine learning techniques. Our approach employs a dimensionality reduction method followed by a classical classifier to categorize galaxies based on shape (cigar/in-between/round; edge-on/face-on) and texture (smooth/featured). We evaluate various dimensionality reduction methods, including Locally Linear Embedding (LLE), Isomap, Uniform Manifold Approximation and Projection (UMAP), t-SNE, and Principal Component Analysis (PCA). Our results demonstrate that most classical classifiers achieve their highest performance when combined with LLE, attaining accuracy comparable to that of simple neural networks. Moreover, in the case of shape classification, the three-dimensional representation remains interpretable, in contrast to the commonly observed loss of interpretability following nonlinear transformations. Additionally, we explore dimensionality reduction followed by k-means clustering to assess whether the data exhibits a natural tendency toward a specific number of clusters. We evaluate clustering performance using silhouette, elbow, Dunn, and Davies-Bouldin scores. While the Davies-Bouldin score indicates a slight preference for four clusters (closely aligning with classifications made by human astronomers) the other metrics do not support a distinct clustering structure.
Figures
Figures from the paper (9 more)
Reference graph
Works this paper leans on
-
[7]
2016.07.03.0114, doi:10.6062/jcis.2016.07.03.0114
URL: http://dx.doi.org/10.6062/jcis. 2016.07.03.0114, doi:10.6062/jcis.2016.07.03.0114. Beck, M.R., Scarlata, C., Fortson, L.F., Lintott, C.J., Simmons, B.D., Galloway, M.A., Willett, K.W., Dickinson, H., Masters, K.L., Marshall, P.J., Wright, D.,
-
[8]
Visualizing the Hidden Features of Galaxy Morphology with Machine Learning
Visualizing the hidden features of galaxy morphology with machine learning. URL: https://arxiv.org/abs/1807.05657, doi:10.48550/ARXIV.1807.05657. Daniel, S.F., Connolly, A., Schneider, J., Vanderplas1, J., Xiong, L.,
-
[10]
Monthly Notices of the Royal Astronomical Society 476, 3661–3676
Improving galaxy morphologies for sdss with deep learning. Monthly Notices of the Royal Astronomical Society 476, 3661–3676. URL: http://dx.doi.org/10.1093/mnras/sty338, doi:10.1093/mnras/sty338. Euclid Collaboration, Bretonni `ere, H., Huertas-Company, M., Boucaud, A., Lanusse, F., Jullo, E., Merlin, E., Tuccillo, D., Castellano, M., Brinch- mann, J., Co...
-
[13]
A comparison of deep learning architectures for optical galaxy morphology classification, in: 2021 International Conference on Electrical, Computer and Energy Technologies (ICECET), IEEE. p. 1–5. URL: http://dx.doi.org/ 10.1109/ICECET52533.2021.9698414, doi: 10.1109/icecet52533. 2021.9698414. Fielding, E., Nyirenda, C.N., Vaccari, M.,
arXiv 2021
-
[14]
The classification of optical galaxy morphology using unsupervised learning techniques. International Conference on Electrical, Computer and Energy Technologies (ICECET) 20-22 July 2022, Prague-Czech , 1–6. 9 Geda, R., Crawford, S.M., Hunt, L., Bershady, M., Tollerud, E., Randria- mampandry, S.,
work page 2022
-
[16]
Manifold learning in the context of quasar spectral diversity. Proceedings of the XIX Serbian Astronomical Conference, October 13-17, 2020, Belgrade, Serbia 100, 241–246. Kalvankar, S., Pandit, H., Parwate, P.,
work page 2020
-
[17]
Galaxy morphology classification using efficientnet architectures. URL: https://arxiv.org/abs/2008. 13611, doi:10.48550/ARXIV.2008.13611. Khlevniuk, A., 2025a. “ellipses explorer” demo, pub- lic github repository. https://mtsizh.github.io/ galaxy-morphology-manifold-learning/demos/generated_ galaxies_explorer.html. Accessed: 2025-03-09. Khlevniuk, A., 202...
-
[19]
URL:https: //arxiv.org/abs/2110.01024, doi:10.48550/ARXIV.2110.01024
Galaxy morphological classification with efficient vision transformer. URL:https: //arxiv.org/abs/2110.01024, doi:10.48550/ARXIV.2110.01024. Linderholm, C.E., Segal, A.C.,
Show all 25 references
-
[20]
Monthly Notices of the Royal Astronomical Society: Letters 477, L101–L105
Gradi- ent pattern analysis applied to galaxy morphology. Monthly Notices of the Royal Astronomical Society: Letters 477, L101–L105. URL: http://dx. doi.org/10.1093/mnrasl/sly054, doi:10.1093/mnrasl/sly054. Shamir, L.,
-
[21]
Early Results From GLASS- JWST. XII. The Morphology of Galaxies at the Epoch of Reionization. The Astrophysical Journal 942, L28. doi: 10.3847/2041-8213/ac9283, arXiv:2207.13527. Tymchyshyn, V .,
-
[24]
URL: https:// arxiv.org/abs/2111.00961, doi:10.48550/ARXIV.2111.00961
Robustness of deep learning algorithms in astronomy – galaxy morphology studies. URL: https:// arxiv.org/abs/2111.00961, doi:10.48550/ARXIV.2111.00961. ´Ciprijanovi´c, A., Kafkes, D., Snyder, G., S ´anchez, F.J., Perdue, G.N., Pedro, K., Nord, B., Madireddy, S., Wild, S.M.,
-
[25]
Machine Learning: Science and Technology 3, 035007
Deepadver- saries: examining the robustness of deep learning models for galaxy morphology classification. Machine Learning: Science and Technology 3, 035007. URL: http://dx.doi.org/10.1088/2632-2153/ac7f1a, doi:10.1088/2632-2153/ac7f1a. Appendix Results of supervised classific...
-
[55]
Fielding, E., Nyirenda, C.N., Vaccari, M.,
URL: http://dx.doi.org/10.1088/0004-637X/ 814/1/55, doi:10.1088/0004-637x/814/1/55. Fielding, E., Nyirenda, C.N., Vaccari, M.,
- [174]
-
[202]
Jankov, I., Ilic, D., Kovacevic, A.,
URL: https://doi.org/10.3847/1538-3881/ac5908, doi:10.3847/1538-3881/ac5908. Jankov, I., Ilic, D., Kovacevic, A.,
-
[284]
Cheng, T.Y ., Huertas-Company, M., Conselice, C.J., Arag ´on-Salamanca, A., Robertson, B.E., Ramachandra, N.,
URL: http://dx.doi.org/10.3847/1538-3881/aae9f4, doi:10.3847/1538-3881/aae9f4. Cheng, T.Y ., Huertas-Company, M., Conselice, C.J., Arag ´on-Salamanca, A., Robertson, B.E., Ramachandra, N.,
-
[2015]
Monthly Notices of the Royal Astronomical Society 450, 1441–1459
Rotation-invariant convolu- tional neural networks for galaxy morphology prediction. Monthly Notices of the Royal Astronomical Society 450, 1441–1459. URL: http://dx. doi.org/10.1093/mnras/stv632, doi:10.1093/mnras/stv632. Dom´ınguez S ´anchez, H., Huertas-Company, M., Bernard...
-
[2018]
Monthly Notices of the Royal Astronomical Society 476, 5516–5534
Integrating human and machine intelligence in galaxy morphology classification tasks. Monthly Notices of the Royal Astronomical Society 476, 5516–5534. URL: http://dx.doi.org/10.1093/mnras/sty503, doi:10.1093/mnras/sty503. Bodi, A., Hajdu, T.,
-
[2019]
Optuna: A next- generation hyperparameter optimization framework, in: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, Association for Computing Machinery, New York, NY , USA. p. 2623–2631. URL: https://doi.org/10.1145/3292500.3...
-
[2020]
Astron- omy and Computing 30, 100334
Machine and deep learning applied to galaxy morphology - a comparative study. Astron- omy and Computing 30, 100334. URL: http://dx.doi.org/10.1016/ j.ascom.2019.100334, doi:10.1016/j.ascom.2019.100334. Barchi, P., da Costa, F., Sautter, R., Rosa, R., Carvalho, R.,
2019
-
[2021]
Monthly Notices of the Royal Astronomical Society 503, 4446–4465
Beyond the hubble sequence – exploring galaxy morphology with unsupervised machine learning. Monthly Notices of the Royal Astronomical Society 503, 4446–4465. URL: http://dx.doi.org/10.1093/mnras/stab734, doi:10.1093/mnras/ stab734. Dai, J.M., Tong, J.,
-
[2022]
Euclid preparation. XIII. Forecasts for galaxy morphology with the Euclid Survey using deep generative models. Astronomy and Astrophysics 657, A90. doi: 10.1051/0004-6361/202141393, arXiv:2105.12149. Ferrari, F., Carvalho, R.R.d., Trevisan, M.,
-
[2023]
Monthly Notices of the Royal Astronomical Society 528, 82–107
Unveiling galaxy morphology through an unsupervised-supervised hybrid approach. Monthly Notices of the Royal Astronomical Society 528, 82–107. URL: http://dx.doi. org/10.1093/mnras/stad3934, doi:10.1093/mnras/stad3934. Lin, J.Y .Y ., Liao, S.M., Huang, H.J., Kuo, W.T., Ou, O.H.M.,
-
[2025]
galaxy morphological classification with mani- fold learning
“galaxy morphological classification with mani- fold learning” public github repository. https://github.com/mtsizh/ galaxy-morphology-manifold-learning . Accessed: 2025-03-09. Vanderplas, J.T., Connolly, A.J.,
2025
-
[5312]
Walmsley, M., Lintott, C., G ´eron, T., Kruk, S., Krawczyk, C., Willett, K.W., Bamford, S., Kelvin, L.S., Fortson, L., Gal, Y ., et al.,
URL: http://dx.doi.org/ 10.21105/joss.05312, doi:10.21105/joss.05312. Walmsley, M., Lintott, C., G ´eron, T., Kruk, S., Krawczyk, C., Willett, K.W., Bamford, S., Kelvin, L.S., Fortson, L., Gal, Y ., et al.,
Reviewed August 11, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.