{"id":"760630ab-9a66-427e-8d92-64328c59f7e3","arxiv_id":"2412.09358","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"LLE plus logistic regression classifies galaxy shape and texture from Galaxy Zoo DECaLS images with accuracy comparable to simple neural networks, and the 3D embedding is interpretable as ellipticity, orientation, and scale.","lead":"This paper tests five dimensionality reduction methods followed by simple classifiers on Galaxy Zoo DECaLS galaxy images, and finds Locally Linear Embedding (LLE) gives the best accuracy, close to simple neural networks. It also shows the 3D LLE embedding of galaxy shapes forms an almond-shaped manifold whose coordinates can be interpreted as galaxy ellipticity, orientation, and size.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Table 5's smooth/featured accuracy (0.85) is contradicted by the paper's own confusion matrix (Figure 8), which implies 0.78; if the lower value is correct, LLE+LR is not comparable to the FCNN (0.83) for texture classification.","rationale":"The reader's weakest_assumption focuses on the Galaxy Zoo label thresholds, but I find the internal inconsistency between Table 5 and Figure 8 to be the single most load-bearing concern: it directly contradicts a headline accuracy number and, if the lower value is correct, invalidates the comparability claim for the smooth/featured task. The reader's rationale already flags this inconsistency, so we partially agree, though our identified weakest point differs. The label threshold issue is plausible but less decisive because it applies to all Galaxy Zoo-based methods and is not specific to this paper's argument. The numerical inconsistency is localized to one task, so the paper may be salvageable if the authors reconcile the numbers and add error bars; the public code makes this check straightforward. Thus the conditional verdict remains appropriate.","tokens_in":14335,"tokens_out":4035,"duration_ms":36584,"concrete_test":"Run the authors' public repository (github.com/mtsizh/galaxy-morphology-manifold-learning) on the smooth/featured task using the stated LLE parameters (components=64, neighbors=96) and the same 20,000-image sample with a 70/30 split. Compare the confusion matrix and classification report to Figure 8 and Table 5. Specifically, recompute overall accuracy as (TP_smooth + TP_featured)/6000 and per-class recall from the matrix; if the accuracy is 0.78, Table 5 is wrong; if 0.85, Figure 8 is wrong. Also run 5-fold cross-validation to report the variance, since the paper currently gives no error bars.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that LLE-based classical pipelines achieve accuracy comparable to simple neural networks rests, for the smooth/featured task, on the 0.85 accuracy reported in Table 5. However, the confusion matrix in Figure 8 shows 2,428 correct smooth plus 2,267 correct featured out of 6,000 test images, i.e., 4,695/6,000 = 0.7825. The per-class precision/recall in Table 5 (smooth 0.86/0.85, featured 0.85/0.85) also cannot be reconciled with that matrix; the matrix gives smooth recall 2,428/3,103 ≈ 0.78 and featured recall 2,267/2,897 ≈ 0.78. If the confusion matrix is accurate, the true accuracy is about 0.78, below the 0.83 of the simple FCNN reported in Table 11, so the 'comparable to simple neural networks' claim fails for texture classification. If Table 5 is accurate, Figure 8 is erroneous. This is an internal inconsistency, not a tuning or labeling choice, and it is directly checkable from the released code. The other two tasks have consistent tables and matrices, so the error is localized; correcting it may preserve the overall claim, but as submitted the paper's headline result contains a demonstrable numerical contradiction.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript presents a systematic comparison of five dimensionality-reduction methods (LLE, Isomap, UMAP, t-SNE, PCA) combined with classical classifiers (logistic regression, SVC, decision tree) for three Galaxy Zoo DECaLS morphology classification tasks. It reports that LLE yields the best downstream accuracy, that LLE plus logistic regression attains accuracy comparable to simple neural networks, and that a 3D LLE/Isomap/PCA embedding of the round/in-between/cigar data is interpretable in terms of ellipse parameters. It also applies LLE followed by k-means clustering and evaluates cluster validity indices. The paper includes public code and interactive demos.","tokens_in":14651,"tokens_out":10974,"duration_ms":92185,"significance":"If the reported performance and interpretability are correct, the paper provides a useful, computationally light, and interpretable alternative to deep networks for survey-scale morphology classification, especially for fast exploratory analysis. The strengths are the breadth of the method comparison, the public code repository, and the synthetic-model construction made available for testing. However, the internal numerical inconsistencies in the main classification tables and the unclear neural-network comparison protocol prevent the central claim from being accepted as stated.","major_comments":[{"comment":"The smooth/featured confusion matrix in Figure 8 is inconsistent with the classification report in Table 5. Figure 8 gives 2428 correct smooth and 2267 correct featured out of 6000 test images, i.e., an accuracy of 0.7825, whereas Table 5 reports accuracy 0.85. In addition, the row sums of Figure 8 are 3081 for smooth and 2919 for featured, while Table 5 lists support 3103 and 2897, and the per-class precision/recall values (0.86/0.85 and 0.85/0.85) cannot be recovered from the matrix. This contradiction directly affects the conclusion that LLE plus logistic regression is comparable to the FCNN (Table 11, accuracy 0.83) for texture classification. The authors must recompute the classification report and correct either the figure or the table before the central claim can be assessed.","section":"Section 3.2, Figure 8, Table 5"},{"comment":"The main text states that the smooth/featured and edge-on/face-on experiments 'use the same parameters as in the previous case' (LLE components 138, neighbors 10, from Table 3), yet Appendix Tables 13 and 14 report fine-tuned parameters for exactly these two tasks, namely LLE 64 components with 96 neighbors and LLE 199 components with 199 neighbors. The manuscript needs to state unambiguously which parameter sets produced Figures 8 and 10 and Tables 5 and 6. If the fine-tuned parameters were used, the sentence about using the same parameters is wrong; if they were not used, the Appendix tables are mislabeled.","section":"Section 3.2 vs Appendix Tables 13-14"},{"comment":"The neural-network classification reports in Tables 7-12 list support 20,000, whereas the LLE plus logistic regression reports in Tables 4-6 list support 6,000. According to Section 2.4, the neural networks are trained on 14,000 images and tested on 6,000, so a test-set report should have support 6,000. The reported 20,000-support numbers appear to refer to the 'different random sample of 20,000 images' used for additional control. As a result, the comparison in Section 3.2 between LLE plus logistic regression and the neural networks may not be on the same held-out test set. Please report the neural-network metrics on the same 6,000-image test set as the LLE pipeline, or demonstrate that the 20,000-sample evaluation is comparable.","section":"Section 2.4 and Appendix Tables 7-12"},{"comment":"The interpretability claim is not quantified. The assertion that in the 3D embedding the azimuthal angle correlates with the third flattening measure, the polar angle with the orientation angle, and the z-component with total intensity is supported mainly by a synthetic model with a hand-chosen profile exponent (-2.5) and a mapping with free scaling constants k1 and k2. The paper should provide quantitative evidence on the real data, e.g., Spearman or Pearson correlation coefficients between the embedding coordinates (after the conical-coordinate transformation) and the ellipse parameters, or the residual of the fitted mapping. Without such numbers, 'correlates' is a qualitative visual statement and the abstract's interpretability claim is not fully established.","section":"Section 4, Eq. (1)"}],"minor_comments":[{"comment":"The sentence 'a single image can belong to several classes if they come from different classification problems, thus the total number of images is just 55137' is confusing; it should say that the curated dataset contains 55,137 unique images because the class-specific subsets overlap across the three classification schemes. Also clarify whether the 'unsupervised' subset is the same as the full curated set.","section":"Section 2.1, Table 1"},{"comment":"The parameter optimization is performed using 5-fold cross-validation on a sample of 5,000 images, while the final evaluation uses a 14,000/6,000 split on 20,000 images. The text should state explicitly whether the optimized parameters were re-selected or fixed for the final run, and whether the reported accuracy is on the held-out test fold.","section":"Section 2.3"},{"comment":"The note that 'the number of components for t-SNE should be greater than three times the perplexity' is inconsistent with Table 2, which fixes components at 2 while allowing perplexity up to 50. Please clarify the constraint or correct the table.","section":"Section 2.3 and Table 2"},{"comment":"The phrase 'the multilayer perceptron (fully-connected neural network, FCNN) exhibits the same performance' is ambiguous; it should specify that the FCNN performance is similar to that of the LLE plus logistic regression pipeline, not to the CNN.","section":"Section 3.2"},{"comment":"The term 'spinoric property' is introduced without definition; explain what it means for the 3D representation and how the half-rotation of the galaxy image was verified.","section":"Section 4"},{"comment":"The claim that four clusters 'closely align' with elliptical, spiral, lenticular, and irregular Hubble types is not directly supported, because the classification tasks in this paper use the round/in-between/cigar, smooth/featured, and edge-on/face-on schemes rather than the four Hubble bins. Please temper or justify this interpretation.","section":"Section 3.3"}],"recommendation":"major_revision","confidential_remarks":"The paper has a valuable empirical scope and the public code base is a genuine plus. However, the internal contradictions in the main classification reports (Tables 4-6 versus Figures 6, 8, 10) are serious and suggest a systematic error in computing precision/recall/support or in plotting the confusion matrices. The comparison with the neural networks also needs a matched test protocol. The interpretability section needs quantitative backing. With these fixed, the paper could be publishable. The novelty is moderate: LLE has been used in astronomy before, but the systematic comparison and the 3D interpretability argument are fresh enough for an astronomy-computing journal."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First thing you should know: this is a useful, mostly honest empirical paper, but it has a demonstrable numerical contradiction in its headline texture-classification result. The smooth/featured confusion matrix in Figure 8 implies 4695/6000 = 0.78 accuracy, while Table 5 reports 0.85, and the support counts disagree too. If the matrix is the true one, LLE+LR gets 0.78, below the 0.83 FCNN they compare against, so the \"comparable to neural networks\" claim fails for the texture task. The other two tasks (round/in-between/cigar and edge-on/face-on) have consistent tables and matrices, so this looks like a localized error, not systemic fraud. But it needs fixing before the central claim is credible.\n\nWhat's genuinely new: the systematic comparison of LLE, Isomap, UMAP, t-SNE, and PCA with classical classifiers on GZ DECaLS is a useful benchmark, and the \"almond\" 3D manifold is a real observation. The interpretability mapping from ellipse parameters (a,b,phi) to embedding coordinates, with the third flattening measure, is a concrete and testable proposal. The synthetic model helps make the point. Code and interactive demos are public, which is good practice.\n\nSoft spots beyond the table/matrix issue: no error bars or repeated runs on different seeds, so we don't know the variance around those accuracies. The interpretability mapping uses free scaling constants k1, k2 and a hand-picked synthetic profile exponent, so the claim that the coordinates have \"physical meaning\" is suggestive rather than proven. The clustering section is honest: only Davies-Bouldin hints at four clusters; the other metrics don't. That's fine.\n\nCitation pattern is fine; they cite Vanderplas & Connolly 2009 and the relevant deep learning literature, and the novelty claim is modest. No red flags.\n\nRecommendation: send this to peer review. With the numbers reconciled and error bars added, it's a solid contribution for the galaxy morphology / ML community. The interpretability result alone is worth refereeing.","headline":"Useful benchmark and a genuinely interesting interpretability result, but the smooth/featured numbers contradict each other and need to be reconciled before the main claim is credible.","tokens_in":15197,"tokens_out":3462,"would_cite":true,"duration_ms":33192,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"On Galaxy Zoo DECaLS images, Locally Linear Embedding followed by a logistic regression classifier reaches accuracies comparable to simple neural networks, and the three-dimensional embedding remains physically interpretable.","keywords":["galaxy morphology","manifold learning","Locally Linear Embedding","dimensionality reduction","Galaxy Zoo DECaLS","interpretable embedding","morphological classification","k-means clustering"],"falsifier":"Evaluate the same LLE-plus-logistic-regression pipeline on Galaxy Zoo DECaLS galaxies whose vote fractions fall between 0.3 and 0.7, the region the current curation discards, and compare accuracy on that ambiguous set with the reported 95 percent. In parallel, measure whether the embedding's azimuthal coordinate tracks the independently computed third flattening measure on synthetic ellipses with known axes. If accuracy collapses on the ambiguous set, or the azimuthal correlation is absent on the synthetic set, the paper's central claims are falsified.","tokens_in":14162,"feed_emoji":"🔭","tokens_out":10586,"duration_ms":91985,"temperature":0.7,"pith_summary":"The paper asks whether classical machine learning, preceded by a good dimensionality-reduction step, can match deep networks at galaxy morphology classification. Using Galaxy Zoo DECaLS images and volunteer labels for shape (round/in-between/cigar, edge-on/face-on) and texture (smooth/featured), it compares Locally Linear Embedding (LLE), Isomap, UMAP, t-SNE, and PCA as front ends for ordinary classifiers. The reported result is that LLE gives the best accuracy no matter which classifier follows it: about 95% weighted F1 on the three shape classes, 85% on smooth/featured, and 96% on edge-on/face-on, comparable to a simple fully connected network. The paper also claims that the three-dimensional LLE space is interpretable, with coordinates tracking flattening, orientation, and scale, and that a synthetic three-parameter ellipse model reproduces the same almond-shaped manifold. If these claims hold, fast, low-cost, inspectable morphology classification on survey images is achievable without pretrained deep models.","feed_headline":"Locally Linear Embedding matches simple neural nets on galaxy shapes","feed_subtitle":"Classical ML with LLE hits 95% on Galaxy Zoo shape classes, and the 3D space stays interpretable.","key_machinery":"The load-bearing object is the Locally Linear Embedding itself: LLE expresses every high-dimensional image as a linear combination of its neighbors and preserves those local reconstruction weights in a low-dimensional space, so nearby images stay nearby. The paper's three-dimensional version of this embedding supplies the interpretable coordinate system, and the interpretation is carried by a closed-form parameterization of elliptical galaxy images that maps the semimajor axis $a$, semiminor axis $b$, and orientation $\\varphi$ to embedded coordinates using the third flattening measure $n=(a-b)/(a+b)$ times $\\cos 2\\varphi$ and $\\sin 2\\varphi$. This map predicts the almond shape of the manifold, explains the spinoric half-rotation, and lets the authors check the real-data embedding against synthetic ellipses.","core_discovery":"The central claim is that Locally Linear Embedding is the dimensionality-reduction method of choice for classical classifiers on galaxy morphology: across the classifiers tested, LLE consistently yields the highest subsequent accuracy, and the margin is achieved with only weak sensitivity to hyperparameters. On a 20,000-image subsample, LLE followed by logistic regression reaches 0.95 weighted F1 for round/in-between/cigar, 0.85 for smooth/featured, and 0.96 for edge-on/face-on, matching a three-layer perceptron and coming within a few points of a small convolutional network. The paper further claims that the three-dimensional LLE embedding of the shape classes is genuinely interpretable rather than a black box: it forms an almond-shaped manifold whose conical coordinates correspond to the galaxy's third flattening measure, orientation angle, and total intensity or scale, and a full revolution of the polar angle induces a half-rotation of the galaxy image. The same structure appears in a synthetic dataset of stretched and rotated ellipses, fitted by the closed-form map $(a,b,\\varphi) \\mapsto (k_1 ab (a-b)/(a+b)\\cos 2\\varphi,\\ k_1 ab (a-b)/(a+b)\\sin 2\\varphi,\\ k_2 ab)$.","pith_inferences":["The almond manifold's two angular coordinates effectively parameterize centered ellipses up to rotation and scale, so the embedding could be reused as a source of rotation- and scale-normalized features for other classifiers.","The $2\\varphi$ factors mean the embedding automatically encodes 180-degree rotational symmetry; this might replace explicit rotation augmentation in shape classifiers.","Because preprocessing crops by Petrosian radius, the z-coordinate labeled as scale may partly encode the crop-window normalization rather than physical galaxy size; separating these would clarify how much interpretability survives outside this dataset.","The 0.8 vote-fraction threshold excludes genuinely ambiguous galaxies, so testing the pipeline on the full vote distribution would show whether the reported accuracy extends to the hardest cases."],"forward_implications":["If LLE is the best front end for classical classifiers, survey-scale morphology pipelines need no deep feature extractor to reach competitive accuracy.","The three interpretable coordinates give a direct handle on physical parameters - flattening, orientation, and scale - without fitting explicit models to each image.","The weak dependence on hyperparameters means the pipeline can be deployed without expensive per-dataset tuning searches.","The absence of a clear cluster structure, except a Davies-Bouldin preference for four clusters, suggests morphology is better represented as a continuum than as separated groups in unsupervised settings.","The modest computational cost makes the approach feasible for large upcoming surveys and for fast unsupervised classification without heavy pretrained models."],"supporting_citations":[{"why":"Supplies the Galaxy Zoo DECaLS images and volunteer vote fractions that define every classification label and the preprocessing target.","marker":"Walmsley et al., 2022"},{"why":"Earlier LLE application to galaxy spectra; the motivating precedent for using LLE as a feature extractor on galaxy images.","marker":"Vanderplas and Connolly, 2009"},{"why":"Provides the Bayesian hyperparameter optimization used to tune every reduction-and-classifier pair, so LLE's dominance is not from hand-picked settings.","marker":"Akiba et al., 2019"},{"why":"Computes the Petrosian r80 radii that set the 120x120 crop size used for all images.","marker":"Geda et al., 2022"},{"why":"Defines the third flattening measure n=(a-b)/(a+b) that the interpretation of the embedding's azimuthal coordinate relies on.","marker":"Linderholm and Segal, 1995"},{"why":"Sets the precedent of training and validating morphology models on synthetic galaxy images, which the paper repeats with its ellipse model.","marker":"Euclid Collaboration et al., 2022"}],"fun_headline_variants":["LLE beats other manifold methods for galaxy shapes","Galaxy shape AI: simple LLE matches neural nets","Interpretable manifold: LLE nails galaxy morphology","Locally Linear Embedding tops galaxy classification","Galaxy Zoo: LLE gives interpretable 3D shape maps"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The comparison depends on Galaxy Zoo volunteer vote fractions, cut at class fraction above 0.8 (or below 0.2 for featured) with more than ten voters, being accurate ground truth for morphology; if those thresholded labels are biased or unrepresentative, the reported accuracies will not generalize to the full survey.","fun_headline_variants_meta":{"raw":{"variants":["LLE beats other manifold methods for galaxy shapes","Galaxy shape AI: simple LLE matches neural nets","Interpretable manifold: LLE nails galaxy morphology","Locally Linear Embedding tops galaxy classification","Galaxy Zoo: LLE gives interpretable 3D shape maps"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00014,"raw_usage":{"total_tokens":1200,"prompt_tokens":1021,"completion_tokens":179,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":637,"completion_tokens_details":{"reasoning_tokens":103}},"tokens_in":637,"tokens_out":179,"duration_ms":2829,"temperature":1.0,"reasoning_tokens":103,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T17:05:49.010658+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Evaluate the same LLE-plus-logistic-regression pipeline on Galaxy Zoo DECaLS galaxies whose vote fractions fall between 0.3 and 0.7, the region the current curation discards, and compare accuracy on that ambiguous set with the reported 95 percent. In parallel, measure whether the embedding's azimuthal coordinate tracks the independently computed third flattening measure on synthetic ellipses with known axes. If accuracy collapses on the ambiguous set, or the azimuthal correlation is absent on the synthetic set, the paper's central claims are falsified.","supporting_citations":[],"review_version":1}