{"id":"ee6307df-64ff-471b-a27a-acb47bab9977","arxiv_id":"2412.15533","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Unsupervised domain adaptation transfers a Galaxy Zoo DECaLS-trained morphology model to BASS/MzLS images, yielding improved classifications and a 248,088-galaxy catalogue.","lead":"This paper applies unsupervised domain adaptation to transfer a galaxy morphology classifier trained on DECaLS images to BASS/MzLS images, improving classification performance and releasing a catalogue of 248,088 galaxies. It shows that a label-free fine-tuning approach can mitigate survey-to-survey data shifts, which is relevant for future wide-field imaging surveys.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The claim that UDA reaches source-domain-level accuracy is not yet controlled: Table 3 compares target-on-BMz against source-on-a-different-DECaLS-validation-set, on only 835 high-vote galaxies, with no error bars, and the large-sample 'validation' uses W+23 model predictions as ground truth.","rationale":"The paper is a useful application of UDA to galaxy morphology transfer, and the released catalogue is a practical contribution. The strongest claim, however, is a quantitative one about reaching source-domain-level performance, and the evidence for it has a control problem. The source-domain baseline is measured on a different validation set, while the target-domain result is measured on a small, high-vote overlap subset with no uncertainty quantification. The high-consistency result against Walmsley et al. (2023) is agreement with another model, not with human labels, and the human-label check in Appendix A actually shows substantially worse performance on detailed spiral features. This does not prove the method fails, but it means the headline parity claim is not yet demonstrated. The reader's stated weakest assumption, equal physical-domain distributions, is plausible given the matched MGS selection cuts, so I do not see it as the main threat. The most load-bearing issue is instead the uncontrolled evaluation: a paired same-galaxy comparison on the available DECaLS stamps of the overlap galaxies would settle whether the claimed improvement is real. The reader already issued CONDITIONAL with moderate confidence, and my concern reinforces that condition rather than overturning the paper, so the verdict should remain UNCHANGED.","tokens_in":16443,"tokens_out":5443,"duration_ms":50839,"concrete_test":"Use the 3618 overlapping galaxies as a paired testbed: run the source model on each galaxy's DECaLS stamp and on its BMz stamp, and run the target model on the BMz stamp, scoring all against the same GZD-5 vote labels using the same 835-galaxy filtering. Report per-question accuracy and F1 with bootstrap 95% confidence intervals, plus a paired McNemar test for source-on-BMz versus target-on-BMz. If target-on-BMz matches source-on-DECaLS on the same galaxies, the parity claim is confirmed; if source-on-DECaLS on this subset already matches target-on-BMz, the headline comparison was confounded by sample selection.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim rests on Table 3. The 3618 overlap galaxies shrink to 835 after vote filtering, and per-question counts are often small (e.g. Bar N=133, Spiral winding N=98). The source-domain baseline in Table 2 is computed on a different DECaLS validation sample from the full survey, not on the DECaLS images of the same 835 galaxies. Thus the target model's 'comparable to source domain' performance could reflect the overlap subset being brighter or easier, rather than successful domain adaptation. No confidence intervals or significance tests are reported; for example, Bar improving from 75.9% to 83.4% at N=133 is roughly a 2-sigma effect, and 'How Rounded' degrades after adaptation. The large-sample consistency check in Table 4 (227,262 galaxies) uses predictions from Walmsley et al. (2023) as reference labels, so it demonstrates agreement with another model, not accuracy against human labels. The only human-label check on a non-trivial sample, Appendix A (N=552), shows Spiral winding accuracy of 65.6% for the target model, well below the claimed source-level ~79% and more than 12 points below the target model's own Table 3 value on the GZD-5 overlap subset; the authors attribute this to volunteer bias, but it also weakens confidence in the parity claim.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper addresses the problem of applying a galaxy morphology classifier trained on DECaLS images (with Galaxy Zoo DECaLS 5 volunteer labels) to BASS/MzLS (BMz) images, which suffer from a distributional shift. The authors train a source-domain EfficientNet-v2-s model on DECaLS images with Dirichlet-multinomial loss, then fine-tune it on 248,088 unlabelled BMz galaxies using an unsupervised domain adaptation (UDA) method based on spherical K-means pseudo-labels and a temperature-scaled contrastive loss. They evaluate the target model on 3,618 overlap galaxies (835 after vote filtering) between DECaLS and BMz, and also compare their predictions against those of Walmsley et al. (2023). They release a catalogue of detailed morphology classifications for 248,088 BMz galaxies. The central claim is that the UDA-fine-tuned target model significantly improves over directly applying the source model to BMz and reaches performance comparable to the source-domain model on DECaLS.","tokens_in":16770,"tokens_out":4132,"duration_ms":22030,"significance":"If the claim holds, the paper demonstrates a label-free route for transferring detailed morphology classifiers across galaxy surveys and provides a practical catalogue for 248,088 BMz galaxies, complementing the Zoobot series. The source-domain model reproduces W+21-level performance, and the evaluation on held-out volunteer labels excluded from training is a genuine strength. The released catalogue, code availability, and use of well-established GZD-5 labels give the work practical value. However, the load-bearing quantitative evidence for the parity claim is weakened by small per-question samples, absent uncertainty estimates, an uncontrolled cross-sample comparison for the 'comparable to source domain' statement, and an unresolved discrepancy with GZD-8 labels in Appendix A. These issues are addressable with additional analysis, but they currently limit the strength of the central conclusion.","major_comments":[{"comment":"The claim that the target model 'reaches a level comparable to that of the source domain' compares the target model on 835 BMz galaxies (Table 3b) with the source model on a different DECaLS validation set (Table 2). This is not an apples-to-apples comparison: the 835-galaxy overlap subset may be unrepresentative (e.g., brighter or higher-signal) of the full DECaLS validation sample. To support the parity claim, the authors should evaluate the source model on the DECaLS images of the same 835 overlap galaxies, or otherwise demonstrate that the overlap subset has the same difficulty distribution as the full validation set. Without this control, the apparent 'parity' could be driven by sample selection rather than successful domain adaptation.","section":"§4.2, Tables 2 and 3"},{"comment":"Many per-question counts are small (Bar N=133, Spiral winding N=98, Edge-on bulge N=16), and no confidence intervals or significance tests are provided. For example, the Bar accuracy improvement from 75.9% to 83.4% on N=133 is roughly a 1.5–2 sigma effect, and 'How Rounded' actually degrades from 90.5% to 87.7% after adaptation. The statement that the target model 'significantly improves performance' is therefore not quantitatively supported for several questions. The authors should report bootstrap confidence intervals or a statistical test for each metric, and discuss the 'How Rounded' degradation in this context rather than dismissing it as an acceptable trade-off.","section":"§4.2, Table 3"},{"comment":"The target model's accuracy on 'Spiral winding' with GZD-8 labels is 65.6%, which is more than 12 points below its Table 3 value on the GZD-5 overlap subset (79.5%) and well below the source-domain value in Table 2 (79.1%). The authors attribute this to volunteer bias in GZD-8, but the same bias argument could apply to the Table 3 ground truth, and the consistency check in Table 4 uses predictions derived from GZD-8 labels. This internal tension undermines the robustness of the claimed parity. The authors should either provide a quantitative analysis of the label-bias hypothesis (e.g., comparing volunteer vote distributions on the same galaxies) or temper the parity claim.","section":"Appendix A, Table A1"},{"comment":"Using Walmsley et al. (2023) predictions as reference labels measures agreement with another model, not accuracy against human labels. The statement that the target model 'does not exhibit significant bias under data shift' is not warranted by agreement with a model whose own accuracy is not established on the same galaxies. Moreover, the comparison in Table 4 again contrasts target-on-BMz with source-on-GZD (Table 2), which confounds domain shift with sample differences. A cleaner check would compare source and target model predictions on the same set of BMz galaxies with the same reference labels, and ideally on a human-labelled subset.","section":"§4.2.1, Table 4"}],"minor_comments":[{"comment":"The paragraph beginning 'Specifically, seven morphology questions...' is repeated verbatim, which should be fixed.","section":"§4.2.1"},{"comment":"The footnote 'assuming a set of concomitant 5 q = 10 questions' is unclear; the authors should clarify what 'concomitant' means here and whether the loss assumes independence or dependence among questions.","section":"§3.1, Eq. (1)"},{"comment":"The tables report counts but not uncertainty intervals; adding bootstrap CIs or at least standard errors for each accuracy value would greatly improve interpretability.","section":"Tables 2 and 3"},{"comment":"The sentence 'We don’t use the prediction results of 314,000 galaxies from W+21' is a footnote and would be clearer in the main text or omitted entirely.","section":"§2.3"},{"comment":"The abstract uses 'domain adaption' while the title and text use 'domain adaptation'; please standardize the spelling.","section":"Abstract and title"}],"recommendation":"major_revision","confidential_remarks":"The paper presents a useful practical resource and a plausible UDA pipeline, but the central parity claim is currently not nailed down by the evidence. The good news is that the issues are fixable: adding source-model evaluation on the same overlap galaxies' DECaLS images, providing bootstrap confidence intervals, and resolving the GZD-8 discrepancy would substantially strengthen the paper. I would not reject, because the catalogue and method are valuable even if the parity claim needs qualification."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear colleague,\n\nThis is a useful and honest application paper. The authors take an off-the-shelf UDA method (Wang et al. 2023), adapt it to the full 34-feature Galaxy Zoo DECaLS decision tree, and release a catalogue of 248,088 BASS/MzLS galaxies with morphology probabilities and variances. The source model performs comparably to Walmsley et al. on DECaLS, and the target model improves over directly applying the source model to BMz on most questions. The evaluation on 3,618 overlap galaxies with volunteer labels is independent of the model's own predictions, and the authors explicitly state the same-physical-domain assumption and discuss its limits. That is real work and a real resource.\n\nThe soft spots are the ones the stress-test note identifies, and they are load-bearing for the headline claim. Table 3 is the only human-label check of the key claim, and after vote filtering it reduces to 835 galaxies, with per-question samples as small as 16. There are no error bars or significance tests, so the 'comparable to source domain' claim is not yet controlled. Worse, the source-domain baseline in Table 2 is computed on a different DECaLS validation sample, not on the DECaLS images of the same 835 overlap galaxies. That means the apparent recovery could reflect the overlap subset being easier, not successful adaptation. The large-sample consistency check in Table 4 uses Walmsley et al. (2023) predictions as reference labels, so it demonstrates agreement with another model, not accuracy. And Appendix A, the only non-trivial human-label check outside the overlap footprint (N=552), shows Spiral winding accuracy at 65.6%, more than 12 points below the target model's own Table 3 value; the authors blame volunteer bias, but that explanation is not established and it weakens the parity claim.\n\nI also note the code is not yet public and the comparison with W+23 is presented with a duplicated paragraph (minor editorial issue). Overall, the central claim needs revision: either restrict the claim to 'improvement over direct transfer' and present the overlap subset results with uncertainty estimates, or compare on the same galaxies with a matched DECaLS validation set.\n\nThis paper deserves peer review rather than desk rejection, because the resource and the testbed are valuable and the failure modes are fixable. I'd send it back for revision, with the request to add error bars/significance tests, redo the baseline comparison on matched images, and temper the parity claim. I wouldn't cite it in my own work until those numbers are tightened.\n\nBest","headline":"Useful UDA application and catalogue, but the headline parity claim rests on a small unmatched test set and needs revision.","tokens_in":17336,"tokens_out":2853,"would_cite":false,"duration_ms":22875,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Unsupervised domain adaptation transfers a detailed galaxy morphology classifier from DECaLS to BASS/MzLS without new labels.","keywords":["galaxy morphology","unsupervised domain adaptation","transfer learning","DESI Legacy Surveys","DECaLS","BASS","MzLS","deep learning"],"falsifier":"Apply the fully trained UDA model to the 3,618 common DECaLS/BMz galaxies and compare accuracy per question to the source model without adaptation; if the UDA model does not substantially beat the source model on questions like 'Bar' and 'Spiral Arm Count', the claimed transfer has not occurred.","tokens_in":16271,"feed_emoji":"🌌","tokens_out":8241,"duration_ms":61264,"temperature":0.7,"pith_summary":"The paper sets out to show that a deep-learning model for detailed galaxy morphology classification, trained on DECaLS images with Galaxy Zoo volunteer labels, can be adapted to BASS/MzLS images without collecting a single new morphology label. The authors achieve this by unsupervised domain adaptation (UDA): they fine-tune the model's feature extractor on unlabeled BMz galaxies, using pseudo-labels from spherical k-means clustering to align the target domain embedding distribution with the fixed source-domain classifier. On the 3,618 galaxies common to both surveys, the adapted model performs significantly better than directly applying the source model, reaching accuracy close to the source model's own performance on DECaLS. If this holds, it would mean that detailed morphology labels can be propagated across surveys with different telescopes and image quality for a modest computational cost.","feed_headline":"No labels needed: galaxy morphology model transfers surveys","feed_subtitle":"Unsupervised domain adaptation lifts BASS/MzLS classification accuracy to DECaLS levels; 248,088 galaxies released.","key_machinery":"The load-bearing mechanism is a pseudo-label alignment loop built on spherical k-means and a fixed classifier. Each epoch, the target-domain feature embeddings are clustered in cosine space; galaxies whose embedding lies close enough to a cluster center (cosine similarity above a schedule threshold) receive the corresponding morphology pseudo-label. The feature extractor is then fine-tuned by a softmax cross-entropy loss (with temperature) that pushes these embeddings onto the fixed source classifier's weight vectors, which also serve as the initial cluster centers. This alignment transfers the source model's decision boundaries to the target domain without requiring any target labels.","core_discovery":"The central claim is that UDA with pseudo-label alignment closes most of the 'data shift' gap between DECaLS and BMz imaging. The authors start from a source model (EfficientNet-v2-s with a Dirichlet-multinomial loss) that matches earlier GZD-5 results. They then freeze the classifier weights and fine-tune the feature extractor on 248,088 unlabeled BMz galaxies, assigning pseudo-labels only to embeddings whose cosine similarity to a spherical k-means cluster center (initialized from the classifier weight vectors) exceeds a threshold that decays during training. On the overlapping-footprint test set of 3618 BMz galaxies that were excluded from training, the UDA model improves accuracy over the direct source model on most questions (e.g., 'Bar' from 75.9% to 83.4%, close to the 82.1% source-domain value), confirming that the transfer works and that the released catalogue of 248,088 classifications is not merely source-model extrapolation.","pith_inferences":["The success of the pseudo-label alignment suggests that a broader class of cross-survey classifiers (e.g., for photometric redshifts or star-galaxy separation) could be transferred the same way, provided the physical domain overlap is controlled.","The probability-simplex analysis indicates that lower-resolution BMz images push predictions toward the center (ambiguous) region; a resolution-aware or multi-resolution augmentation might recover some of the tail of the distribution.","If the UDA model's pseudo-labels are reliable enough, one could use them as training labels in a second round of supervised fine-tuning, potentially reducing the remaining gap on detailed features like spiral winding.","The requirement that the two surveys share the same physical galaxy population implies the method cannot be blindly applied to fainter or higher-redshift samples; a domain-discriminator check could quantify when the assumption breaks."],"forward_implications":["If the UDA transfer works, the released catalogue provides detailed morphology probabilities for 248,088 BASS/MzLS galaxies, enabling studies of bar fraction, bulge size, and spiral structure in the northern DESI footprint.","The pipeline can be rerun for other surveys that share a physical galaxy population with DECaLS, such as future wide-field imaging, without waiting for volunteer labels.","The two-step training (source model then label-free fine-tune) means that new surveys need only unlabeled images and a matched source catalogue to obtain per-question morphology predictions.","Because the classifier weights are shared between domains, the target model is directly comparable with the source model, giving consistent morphology definitions across surveys."],"supporting_citations":[{"why":"Provides the GZD-5 volunteer morphology votes and the Dirichlet-multinomial loss used to train the source domain model.","marker":"Walmsley et al. (2021)"},{"why":"Supplies the unsupervised domain adaptation method (pseudo-label alignment via spherical k-means) adapted in this work.","marker":"Wang et al. (2023)"},{"why":"Shows systematic latent-space differences between DECaLS and BMz galaxies, establishing the data shift the UDA method targets.","marker":"Xu et al. (2023)"},{"why":"Gives independent morphology predictions for BMz galaxies (from GZD-8) used to verify the target model's consistency.","marker":"Walmsley et al. (2023)"}],"fun_headline_variants":["No-label AI maps 248K galaxies across surveys","Survey-hopping morphology model beats direct transfer","Unsupervised adaptation closes survey gap in galaxy shapes","Galaxy morphology model crosses surveys without new labels","BMz galaxies get DECaLS-grade morphology from UDA"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes that the DECaLS and BASS/MzLS galaxies are drawn from the same physical galaxy population, so aligning their feature embeddings with pseudo-labels is meaningful.","fun_headline_variants_meta":{"raw":{"variants":["No-label AI maps 248K galaxies across surveys","Survey-hopping morphology model beats direct transfer","Unsupervised adaptation closes survey gap in galaxy shapes","Galaxy morphology model crosses surveys without new labels","BMz galaxies get DECaLS-grade morphology from UDA"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000502,"raw_usage":{"total_tokens":2504,"prompt_tokens":1045,"completion_tokens":1459,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":661,"completion_tokens_details":{"reasoning_tokens":1384}},"tokens_in":661,"tokens_out":1459,"duration_ms":8703,"temperature":1.0,"reasoning_tokens":1384,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T11:20:26.158627+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Apply the fully trained UDA model to the 3,618 common DECaLS/BMz galaxies and compare accuracy per question to the source model without adaptation; if the UDA model does not substantially beat the source model on questions like 'Bar' and 'Spiral Arm Count', the claimed transfer has not occurred.","supporting_citations":[{"cited_title":"Multimedia] 10.1109/TMM.2022.3146744 , 25, 1665","cited_arxiv_id":null,"evidence_quote":"Supplies the unsupervised domain adaptation method (pseudo-label alignment via spherical k-means) adapted in this work."}],"review_version":1}