{"id":"d05b9f9b-deea-4145-abc3-4509f515374d","arxiv_id":"2501.09816","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Zernike moments with a support vector machine classify clean Galaxy Zoo 2 galaxy images into spiral, elliptical, and odd classes with accuracy above 0.90 and TSS above 0.86.","lead":"This paper tests whether Zernike moments, a compact set of one-dimensional shape features, can classify galaxy images as galaxies versus non-galaxies and as spirals, ellipticals, or odd objects. It reports that a support vector machine on 1081 Zernike features reaches accuracy above 0.90 and TSS above 0.86 on clean Galaxy Zoo 2 samples, comparable to deep image models at lower cost.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Test-set selection of p_max inflates the reported TSS; an independent validation split is needed before the headline performance claim is accepted.","rationale":"The reader's weakest assumption already identifies the pmax selection issue, and my analysis agrees. This is the most load-bearing condition because the central claim is a quantitative performance assertion; if the performance estimate is selected rather than confirmed, the main conclusion that ZMs are high-performance descriptors lacks unbiased support. I also note that the abstract's statement that 1D-CNN has TSS greater than 0.86 for the oversampled binary classifier conflicts with the body's reported 1D-CNN TSS of 0.70, which further motivates a clean evaluation protocol. The concern is moderately severe but addressable, so it does not require rejection; it does require verification with an independent holdout or nested cross-validation. Hence the reader's CONDITIONAL verdict is appropriate.","tokens_in":23918,"tokens_out":7897,"duration_ms":75546,"concrete_test":"Hold out a fixed 20% test set before any model selection. On the remaining 80%, perform cross-validation over pmax in {5,...,46} and choose the pmax maximizing mean validation TSS for each classifier. Retrain with that pmax and evaluate once on the held-out test set. Report test TSS and accuracy for SVM and 1D-CNN on both the binary and three-class tasks. If the held-out SVM or 1D-CNN binary TSS falls below 0.86, the headline performance claim is not corroborated.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central performance estimates are not unbiased because the maximum Zernike order was chosen by evaluating accuracy on the same test split later used to report the headline metrics. Section 4.2 states: 'Examining the ZMs for maximum order number (pmax) ranges from 5 to 46, we calculate the performance metric (e.g., accuracy) for both the SVM and classic 1D-CNN classifiers. We obtain the highest performance for pmax=45...' With roughly 42 candidate pmax values, selecting the best on the test set can overstate TSS and accuracy even when each split is averaged over ten random repetitions, because the selection criterion is the same test set. This affects both the binary galaxy/non-galaxy numbers (SVM TSS=0.86, 1D-CNN TSS=0.70) and the three-class numbers (SVM TSS=0.88, 1D-CNN TSS=0.86). The problem is not the descriptive power of Zernike moments; it is the lack of an independent estimate of generalization for the specific pmax=45 choice. A separate validation set held out before the pmax scan is required to support the abstract's 'TSS greater than 0.86' claim.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes using Zernike moments (ZMs) of galaxy images as compact, rotation-invariant features for morphological classification, and compares five models: SVM with ZMs, 1D-CNN with ZMs, and three image-based deep networks (2D-CNN, ResNet50, VGG16 with vision transformer). The data are drawn from GalaxyZoo 2. Two classification tasks are considered: galaxy vs. non-galaxy, and three-class galaxy morphology (spiral, elliptical, odd). The central claims are that ZM-based SVM and 1D-CNN achieve high performance (TSS > 0.86) on the galaxy/non-galaxy task, and that SVM with ZMs, 1D-CNN with ZMs, and VGG16 achieve accuracy > 0.90 and TSS > 0.86 on the three-class task. The authors also argue that the ZMs provide uniqueness, completeness, and invariances, and that the ZM-based methods have lower computational cost than deep image models.","tokens_in":24172,"tokens_out":4535,"duration_ms":40772,"significance":"If the reported performance estimates were unbiased, the paper would make a useful practical contribution: 1081 Zernike moments can match or approach deep image classifiers on a moderately clean Galaxy Zoo 2 sample while using far fewer parameters and less computational resources. The authors provide code and data on GitHub, which supports reproducibility. However, the headline performance numbers are currently undermined by a model-selection procedure that uses the test set to choose the maximum Zernike order p_max, and by an abstract that states TSS values inconsistent with the body. These issues are central to the paper's main claim and must be fixed before the results can be accepted as reliable.","major_comments":[{"comment":"The maximum Zernike order p_max was selected by evaluating accuracy on the same test set later used to report all performance metrics. The text states: 'Examining the ZMs for maximum order number (pmax) ranges from 5 to 46, we calculate the performance metric (e.g., accuracy) for both the SVM and classic 1D-CNN classifiers. We obtain the highest performance for pmax = 45.' With roughly 42 candidate values, choosing the best on the test set overstates the generalization performance of the final model, even if the training/test split is repeated ten times, because the selection criterion uses test-set labels. This directly affects the headline binary TSS values (SVM 0.86, 1D-CNN 0.70) and the three-class TSS values in Table 4. Please restructure the evaluation so that p_max is selected on an independent validation set (or via nested cross-validation) and the reported metrics are computed on a held-out test set that is not used at any stage of model selection.","section":"Section 4.2 (and Section 4.3 if p_max is scanned there too)"},{"comment":"The abstract claims 'TSS greater than 0.86 for the SVM and 1D-CNN with ZMs for the oversampled galaxy-non-galaxy classifier,' but the body reports, for the oversampled binary task, SVM TSS = 0.86 +/- 0.01 and 1D-CNN TSS = 0.70 +/- 0.03. The 1D-CNN value is far below 0.86, and the SVM value is not strictly greater than 0.86 given the uncertainty. This is a factual discrepancy between the abstract and the results. Please correct the abstract and ensure all numeric claims match the tables and text.","section":"Abstract and Section 4.2"},{"comment":"The text states that 'TSS is greater than 0.86 for Models I, II, and V' and the abstract says 'accuracy larger than 0.90 and TSS greater than 0.86' for SVM with ZMs, 1D-CNN with ZMs, and VGG16. Table 4 lists TSS = 0.88 +/- 0.003 (Model I), 0.86 +/- 0.01 (Model II), and 0.89 +/- 0.005 (Model V), and accuracy = 0.90 +/- 0.005, 0.90 +/- 0.01, and 0.93 +/- 0.003. For Model II, TSS is not strictly greater than 0.86, and for Models I and II accuracy is not strictly larger than 0.90. Please use phrasing consistent with the uncertainties, e.g., 'TSS at least 0.86' or report lower bounds from the confidence intervals.","section":"Section 4.3, Table 4"}],"minor_comments":[{"comment":"The reconstruction formula I_R(r,theta) = sum_p sum_q Z_q^p ZP_q^p(r,theta) should explicitly include the complex conjugate of the Zernike polynomial or state that the real part is taken; as written, the sum over positive and negative q may not be manifestly real for a real image.","section":"Equation (6)"},{"comment":"The oversampling procedure adds ZMs of the R, G, and B channels of the same non-galaxy images to the training set, but the test set contains only grayscale ZMs. Please clarify in the text that these augmented samples are deterministic transformations of the original minority samples and therefore do not add independent information; this affects how the uncertainty estimates and the improved TSS should be interpreted.","section":"Section 4.2, oversampling discussion"},{"comment":"There are duplicate reference entries: Shamir 2009 appears twice, and Li et al. 2022 appears three times. Please consolidate the bibliography.","section":"Appendix/references"},{"comment":"Minor wording issues: 'different architects' should be 'different architectures'; the GitHub URL in the text contains spaces and should be a proper hyperlink; and the phrase 'the task answer of GZ2' in Section 6 should be rephrased.","section":"Section 3.3 and captions"}],"recommendation":"major_revision","confidential_remarks":"The paper's core idea is reasonable and the experiments are transparent enough to be repairable, but the test-set-based selection of p_max and the abstract/body discrepancy are load-bearing. I recommend major revision rather than rejection, because the authors can re-run the analysis with a proper validation split and correct the reported numbers. The paper would benefit from a more careful statement of uncertainty in all headline claims."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper is worth reading, and the core idea is sound: Zernike moments are a compact, rotation-invariant description of galaxy images, and feeding 1081 of them to an SVM or a 1D-CNN gives accuracy above 0.90 and TSS above 0.86 on clean Galaxy Zoo 2 three-class samples, competitive with VGG16 with a vision transformer. That result, if confirmed, is cheap and practical for big surveys. The analytic section linking the Sersic profile to vanishing even-order Zernike moments is a genuinely nice touch, and the PDF decision-boundary analysis is a simple way to show the features carry class information without any learning model.\n\nThe problems are real but addressable. First, the abstract says TSS greater than 0.86 for both SVM and 1D-CNN in the oversampled galaxy/non-galaxy binary task, but Table 3 in the body reports 1D-CNN TSS = 0.70. The three-class claims for Models I, II, and V are consistent with the abstract, but the binary 1D-CNN claim is not. Second, and more important, the maximum Zernike order p_max was chosen by scanning 5 to 46 and keeping the value with highest accuracy on the same test split later used for the reported metrics. That selection-on-the-test-set inflates the generalization estimate. This is not a fatal flaw, because the descriptive power of Zernike moments is not in question, but it means the honest headline is \"works well on this data set,\" not \"generalizes at TSS > 0.86.\" A quick fix is to hold out a validation set before the p_max scan and report the test metrics only for the chosen p_max. Finally, the GitHub links in the abstract and text contain spaces and are malformed; that is minor but should be fixed.\n\nWho benefits? Anyone working on fast morphological classification for large surveys, and people who want a concrete worked example of Zernike moments applied to a modern data set. The paper does not need to be a breakthrough to be useful, and the methods section is clear enough to reproduce once the links are fixed. My recommendation: send it to peer review, but require the authors to (a) correct the abstract/body inconsistency, (b) rerun the p_max selection with a proper validation split, and (c) fix the code links. Then the claimed numbers will be trustworthy and the contribution will stand.","headline":"A useful, genuinely new application of Zernike moments to optical galaxy classification, but the headline TSS numbers are inflated by test-set model selection and the abstract overstates the 1D-CNN binary result.","tokens_in":24739,"tokens_out":1280,"would_cite":false,"duration_ms":14364,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Zernike moments classify galaxy morphology with accuracy above 0.90 on Galaxy Zoo 2 samples.","keywords":["galaxy morphology","Zernike moments","machine learning classification","support vector machine","convolutional neural network","Galaxy Zoo 2","rotation invariance","imbalanced classification"],"falsifier":"Re-run the SVM and 1D-CNN pipelines with $p_{\\max}$ chosen by cross-validation on training data only, then evaluate on a held-out test set untouched during model selection; if the held-out TSS for the binary classifier falls below 0.86, or the three-class accuracy falls below 0.90, the paper's high-performance claim as stated is not supported.","tokens_in":23691,"feed_emoji":"🔭","tokens_out":9356,"duration_ms":86153,"temperature":0.7,"pith_summary":"The paper tries to show that a compact vector of Zernike moments—coefficients of a galaxy image projected onto an orthogonal polynomial basis on the unit disc—carries enough morphological information to classify galaxies by machine learning. Using Galaxy Zoo 2 images, the authors build a galaxy-versus-non-galaxy classifier and a three-class spiral/elliptical/odd classifier, and they compare Zernike-feature models (SVM and one-dimensional CNN) against image-based deep networks (2D-CNN, ResNet50, VGG16 with a vision transformer). Their central claim is that the Zernike-based models are high-performance: TSS above 0.86 for the oversampled binary SVM, and accuracy above 0.90 with TSS above 0.86 for the SVM, 1D-CNN, and VGG16 models in the three-class task. If true, 1081 rotation-invariant numerical features per clean galaxy image can replace much heavier image-processing pipelines at lower computational cost, which would be useful for upcoming large surveys.","feed_headline":"1081 Zernike moments match deep learning for galaxy sorting","feed_subtitle":"Compact rotation-invariant features reach accuracy above 0.90 and TSS above 0.86 on Galaxy Zoo 2.","key_machinery":"The central object is the Zernike moment: the coefficient $Z^q_p = \\frac{p+1}{\\pi}\\int I(r,\\theta) ZP^{q*}_p \\, r\\,dr\\,d\\theta$ obtained by projecting the image intensity onto the complete orthogonal Zernike polynomials $ZP^q_p(r,\\theta)=R^q_p(r)e^{iq\\theta}$ on the unit disc. Because of the Fourier factor $e^{iq\\theta}$, the magnitudes of the ZMs are invariant under image rotation, and orthogonality and completeness guarantee that the coefficient list is unique and can reconstruct the image. The paper uses the first 1081 moments ($p_{\\max}=45$) per grayscale galaxy as the feature vector for an RBF-kernel SVM and a 1D-CNN, and contrasts this with activation-map features from 2D-CNN, ResNet50, and VGG16 models. The machinery also includes watershed segmentation of galaxy images to a centered 200x200 sub-image (giving translation and scale normalization) and oversampling of the minority non-galaxy class with RGB-channel ZMs.","core_discovery":"The authors' discovery claim is that Zernike moments (ZMs), taken to maximum order $p_{\\max}=45$, give 1081 unique, rotation-invariant descriptors that are directly usable for morphological classification. For the binary task they use 545 non-galaxies and 11,735 galaxies selected by Galaxy Zoo 2 fraction thresholds, oversample the non-galaxy minority by adding the ZMs of the red, green, and blue channels (normalized by the total-intensity moment $Z^0_0$) to reach an imbalance ratio of about 0.2, and report SVM accuracy of 0.90 with TSS of 0.86 and AUC 0.93. For the three-class task, after segmenting images to 200x200 galaxy sub-images, they report weighted TSS of 0.88 for SVM with ZMs, 0.86 for 1D-CNN with ZMs, and 0.89 for VGG16 with a vision transformer, with accuracies 0.90, 0.90, and 0.93, matching or approaching the deep image models while using far fewer features.","pith_inferences":["Editorial inference: the same 1081-moment representation could be tested for continuous structural regressions, since the paper shows that the $q=0$ moments are directly tied to the axisymmetric S\\'ersic-like brightness profile; if the link holds, ZMs might estimate structural parameters rather than only discrete classes.","Editorial note: the abstract says TSS exceeds 0.86 for both binary ZM models, but the body table lists TSS=0.70 for the 1D-CNN binary model; the three-class TSS values quoted in the abstract and table agree, so readers comparing models should use the table values.","Editorial inference: transfer to other surveys is untested; retraining on GZ2 and scoring DECaLS, DES, or Euclid images would show whether the rotation invariance and compactness survive differences in seeing, depth, and bandpass.","Editorial inference: hybridizing Zernike moments with a small CNN could combine the interpretable, low-cost geometric description with deep texture features, possibly improving the odd-object class where the reported recall is lowest."],"forward_implications":["A single 1081-component Zernike vector is enough to separate galaxy from non-galaxy images and spiral, elliptical, and odd galaxies at the accuracy levels reported, without training a deep network on the pixels.","Rotation invariance removes the need to align or rotationally augment galaxy images before classification, simplifying pipelines for surveys where orientation is arbitrary.","Because the 1D-CNN and SVM operate on one-dimensional moment vectors, the trained classifiers run on CPU resources, cutting the GPU cost of ResNet50- and VGG16-style pipelines.","For the three-class problem the ZM-based models are competitive with a VGG16 plus vision transformer on clean Galaxy Zoo 2 samples, suggesting ZMs are a viable low-cost feature set for large morphological samples.","Performance is class-dependent: odd objects have the lowest recall (0.84 for SVM with ZMs), so the practical use needs to weigh the odd-class error against the computational savings."],"supporting_citations":[{"why":"Defines the Galaxy Zoo 2 catalog and the volunteer-response fractions whose thresholds select the spiral, elliptical, and odd-object samples.","marker":"Willett et al. 2013"},{"why":"Provides the gz2hart16 table with per-galaxy task fractions used to apply the class-selection thresholds.","marker":"Hart et al. 2016"},{"why":"Gives the rotation-invariance property of Zernike-moment magnitudes that the feature descriptor relies on.","marker":"Khotanzad & Hong 1990"},{"why":"Introduces Zernike moments as image descriptors, grounding the moment computation in Equation (4).","marker":"Teague 1980"},{"why":"Supplies the orthogonal complete-basis framework that guarantees uniqueness and reconstruction used for the claim that ZMs are independent features.","marker":"Mukundan & Ramakrishnan 1995"},{"why":"Provides the ZEMO implementation used to compute ZMs and reconstructed images in the paper.","marker":"Safari et al. 2023"},{"why":"Foundation of the SVM hyperplane and margin optimization used in Model I.","marker":"Cortes & Vapnik 1995"},{"why":"Defines the VGG16 architecture that serves as the strongest image-based comparison model.","marker":"Simonyan & Zisserman 2014"},{"why":"Introduces the vision transformer whose augmentation and feature-extraction pipeline is appended to the image-based deep models.","marker":"Dosovitskiy et al. 2020"}],"fun_headline_variants":["Zernike moments rival deep nets for galaxy classification","1081 Zernike features match deep learning on galaxy morphologies","Rotation-invariant Zernike moments hit deep-net accuracy on galaxies","Compact Zernike descriptors match CNNs for galaxy sorting"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The reported scores come from a test set that was also used to choose the maximum Zernike order ($p_{\\max}=45$), so the headline accuracy and TSS may be optimistic for images the model has not been tuned on.","fun_headline_variants_meta":{"raw":{"variants":["Zernike moments rival deep nets for galaxy classification","1081 Zernike features match deep learning on galaxy morphologies","Rotation-invariant Zernike moments hit deep-net accuracy on galaxies","Compact Zernike descriptors match CNNs for galaxy sorting"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000771,"raw_usage":{"total_tokens":3507,"prompt_tokens":1130,"completion_tokens":2377,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":746,"completion_tokens_details":{"reasoning_tokens":2307}},"tokens_in":746,"tokens_out":2377,"duration_ms":17202,"temperature":1.0,"reasoning_tokens":2307,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T19:38:20.146498+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the SVM and 1D-CNN pipelines with $p_{\\max}$ chosen by cross-validation on training data only, then evaluate on a held-out test set untouched during model selection; if the held-out TSS for the binary classifier falls below 0.86, or the three-class accuracy falls below 0.90, the paper's high-performance claim as stated is not supported.","supporting_citations":[{"cited_title":"W., Lintott, C","cited_arxiv_id":null,"evidence_quote":"Defines the Galaxy Zoo 2 catalog and the volunteer-response fractions whose thresholds select the spiral, elliptical, and odd-object samples."},{"cited_title":"E., Bamford, S","cited_arxiv_id":null,"evidence_quote":"Provides the gz2hart16 table with per-galaxy task fractions used to apply the class-selection thresholds."},{"cited_title":"1990, IEEE Transactions on Pattern Analysis and Machine Intelligence, 12, 489","cited_arxiv_id":null,"evidence_quote":"Gives the rotation-invariance property of Zernike-moment magnitudes that the feature descriptor relies on."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Introduces Zernike moments as image descriptors, grounding the moment computation in Equation (4)."},{"cited_title":"1995, Pattern Recognition, 28, 1433","cited_arxiv_id":null,"evidence_quote":"Supplies the orthogonal complete-basis framework that guarantees uniqueness and reconstruction used for the claim that ZMs are independent features."},{"cited_title":"2023, Iranian Journal of Astronomy and Astrophysics, 10, 267","cited_arxiv_id":null,"evidence_quote":"Provides the ZEMO implementation used to compute ZMs and reconstructed images in the paper."}],"review_version":1}