{"id":"3700b9b1-a59a-4a85-817b-68cf844b2559","arxiv_id":"2411.18206","paper_version":1,"verdict":"UNVERDICTED","confidence":"HIGH","novelty_score":0.0,"correctness_risk":"low","formal_verification":"none","parameter_count":2,"one_line_summary":"A review of the authors' ULISSE transfer-learning work, in which ImageNet-pretrained EfficientNet features are used to search, classify, and flag astronomical images without retraining on labeled astronomy data.","lead":"This proceedings paper recaps the authors' ULISSE framework, which uses an ImageNet-trained neural network to turn astronomical images into feature vectors for similarity search. It surveys label-free applications to AGN detection, galaxy property estimation, lensing search, and time-series anomaly flagging, but presents no new measurements.","discovery_kind":"review","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central transfer-learning claim is vulnerable to a photometric-proxy confound: feature-space distances may rank galaxies by apparent brightness, size, and color rather than by astrophysical similarity, so the ImageNet-pretrained encoder may not be the operative component.","rationale":"The reader's weakest_assumption correctly identifies the feature-space similarity premise as the weakest link. My concern sharpens that point: the method may be succeeding for trivial photometric reasons, which would mean the ImageNet pretraining is not actually load-bearing. The paper itself is a recap of prior work and does not include the primary evaluations, so no new evidence is presented that would rule out this confound. I am not claiming the method fails; I am claiming the paper's argument, as written, does not exclude the photometric-proxy explanation. An ablation that normalizes out brightness, size, and PSF would settle whether the transferable visual features are the active ingredient. Since the reader already assigned UNVERDICTED and this concern does not move the verdict in a different direction, UNCHANGED is the appropriate recommendation. My agreement is partial rather than full because the reader framed the risk as 'transfer might fail on noisy survey data,' while I would frame it as 'the reported successes may not actually require transfer at all.'","tokens_in":5534,"tokens_out":4882,"duration_ms":50305,"concrete_test":"Re-run the Section 3.2 galaxy-property retrieval on the SDSS DR8/galSpec sample after preprocessing: (1) subtract background and normalize each three-band cutout to unit total flux, and (2) additionally resample to a fixed angular scale or PSF. If the '<1 dex for more than 80% of targets' result survives both ablations, the feature-space similarity is not merely photometric scaling; if it degrades, the reported accuracy is largely attributable to apparent brightness and size, and the transfer-learning claim is unsupported by the presented evidence. A complementary check is to run k-nearest-neighbour retrieval directly on (g-r, r-i, m_r) photometry and compare its SFR/M* accuracy to ULISSE's; if the simple photometric baseline matches ULISSE, no ImageNet-specific transfer is needed to explain the results.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 2.1 defines the method as turning astronomical images into a 1280-dimensional feature space, and Section 2.2 asserts that this 'transforms the problem of analyzing a complex astronomical image into a geometric problem in a high-dimensional feature space' where distances rank objects by similarity. The load-bearing step is that distance in this space corresponds to astrophysical similarity.\n\nThe evidence in Sections 3.1 and 3.2 is equally consistent with a much weaker mechanism: the g,r,i cutouts fed to EfficientNet encode apparent magnitude, angular size, and color. In Section 3.2 the reported metric is the distance between a target's SFR/stellar mass and the averaged value of retrieved neighbours; if neighbours are selected mainly by brightness and redshift, the stated '<1 dex for more than 80% of targets' does not demonstrate transfer learning, only photometric proximity. No ablation is reported that controls for total flux, background subtraction, PSF, or angular scale, and no comparison is made to a baseline k-nearest-neighbour method on simple photometry. The one application with a quantitative simulated-to-real gap, strong lensing in Section 3.3, also cautions against overgeneralizing the transfer claim.\n\nBecause the paper explicitly presents this method as broadly useful for AGN detection, galaxy parameter estimation, lens finding, and anomaly detection, the unconfounded question is whether the ImageNet-pretrained features contribute anything beyond low-level photometric similarity.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This manuscript is a short proceedings paper that describes ULISSE, a transfer-learning framework built on a frozen ImageNet-pretrained EfficientNet feature extractor. The authors' method maps multi-band (g,r,i) astronomical cutouts into a 1280-dimensional feature space and uses distances in that space for nearest-neighbour similarity search. Four applications are summarized: AGN candidate detection in SDSS, estimation of galaxy stellar mass and star formation rate, strong gravitational lensing candidate search, and anomaly/artifact identification in image and time-series data. The central claim is that no labeled training is needed: labels are used only for assessment, and the same geometric feature space serves multiple downstream tasks. The paper reports summary statistics from the authors' previous ULISSE papers (33-60% AGN candidate rates, 20% false positives, <1 dex distance for more than 80% of galaxies, 94%/93% lensing purity) and notes an ongoing simulated-to-real gap for lensing.","tokens_in":5793,"tokens_out":4169,"duration_ms":35847,"significance":"If the central claim is correct, this would be a valuable and economical tool for survey-era astronomy because it promises useful astronomical inference from a frozen extractor without task-specific labeled training, which is especially relevant for LSST-scale data. The paper is commendably explicit about the role of labels (assessment only) and about the simulated-to-real gap in lensing, and the underlying architecture is easy to reproduce. The significance, however, is conditional: the evidence offered is a set of summary statistics quoted from the authors' own papers, with no baseline comparisons, no error bars, and no ablation isolating the contribution of ImageNet-pretrained features from low-level photometric properties. The paper therefore currently functions better as a research programme statement than as a demonstration that the method works as claimed.","major_comments":[{"comment":"The load-bearing assertion that distances in the 1280-dimensional EfficientNet feature space rank galaxies by astrophysical similarity is not established, because no control is reported for the photometric proxies that the input g,r,i cutouts directly encode (total flux, angular size, colour, PSF, background subtraction). The galaxy-property results in Section 3.2 are equally consistent with a k-nearest-neighbour regressor on simple photometry; the reported '<1 dex for more than 80%' does not by itself demonstrate that transfer learning contributes anything beyond low-level photometric proximity. Please add an ablation using photometric features (magnitude, size, colours) as an explicit baseline, or demonstrate that feature-space neighbours remain astrophysically similar after controlling for these quantities.","section":"Section 2.1 and Section 3.2"},{"comment":"The AGN detection evidence is quoted as 'on average higher than 33%' and '20% false positives', but the sample sizes, definitions of 'true' non-AGN, and the retrieval protocol (number of neighbours, threshold) are not given here, and no comparison is made to standard AGN colour-colour or emission-line selection. As written, the claim that the method is 'really competitive' cannot be evaluated from the numbers reported. Please give the complete contingency table or refer the reader to specific tables in [10] and restate the corresponding metrics.","section":"Section 3.1"},{"comment":"The only application with a quantitative simulated result (94% purity in the first 100 candidates and 93% in the first 200) is also the one where the authors state that 'the transition to real observational data has presented additional challenges'. This tension needs to be addressed explicitly in the conclusions: currently the abstract and Section 3.6 generalize across applications as 'successful implementations', which overstates what the paper itself reports. Please either report quantitative results on real lensing candidates or explicitly limit the claim to simulated data.","section":"Section 3.3"},{"comment":"The anomaly-detection applications are described without any quantitative evaluation: 'great potential', 'very few good epochs lost', and 'a lot of problematic images' are not measurable claims. Since the full studies are available ([20] and Fig. 4), please include the relevant evaluation metrics (e.g., precision/recall for flagged epochs, number of images inspected) or state clearly that these are proof-of-concept results with no quantitative validation.","section":"Section 3.4 and Section 3.5"}],"minor_comments":[{"comment":"'classificator' should be 'classifier'.","section":"Section 2.1, footnote"},{"comment":"'Top forthy' should be 'Top forty'.","section":"Figure 3 caption"},{"comment":"'sosia' is an unusual non-standard term; use 'counterparts' or 'similar images'.","section":"Section 3.3"},{"comment":"References [11] and [15] lack author names and are formatted inconsistently; update to standard journal format.","section":"References"},{"comment":"Figure captions say 'See [10] for further details' for reproduced figures; if figures are taken from other papers, permission and attribution should be stated.","section":"Figure captions"},{"comment":"The text says 'the averaged one for the set of the retrieved neighbours' but Fig. 2 also refers to a 'weighted mean'; define both averaging procedures and explain why they differ.","section":"Section 3.2"}],"recommendation":"major_revision","confidential_remarks":"This is a proceedings-style contribution, so the bar for completeness is lower than for a full research paper. Still, the evidence base is a set of self-citations, and the central generalization ('labels can be used just to assess the results') is not supported without a photometric baseline. I would encourage the editor to send the manuscript back with the request for an explicit baseline or ablation even if brief, or to accept a version with suitably conditional wording. The paper should also be checked for the missing author information in references, which may be an editorial issue."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThis is a proceedings-style recap, and the authors say so themselves in the intro: 'we recap some of our transfer learning applications in astronomy.' There is no new derivation, no new data, and no new experiment. If you are looking for a novel contribution, this is not it. The only fresh-looking element is a qualitative figure of 'weird' SDSS images, which has no metrics.\n\nWhat the paper does well is summarize a coherent line of work built around a fixed ImageNet-pretrained EfficientNet feature extractor used for retrieval across four tasks: AGN candidacy, galaxy SFR/stellar mass estimation, strong-lens ranking, and time-series epoch flagging. The method description is clear, and the authors are honest about which results come from which prior paper. For someone wanting an entry point to ULISSE, this is a useful map.\n\nThe soft spots are the ones you would expect in a recap. The load-bearing assertion—that distance in the 1280-dimensional feature space corresponds to astrophysical similarity—is not defended here beyond a citation to [10]. The stress-test concern is real: with g,r,i cutouts, neighbors may be retrieved mainly on apparent brightness, size, and color, and a k-NN baseline on simple photometry would be a fair comparison. But that is a critique of the underlying method, not of this recap, because no new evidence is presented. The one quantitative result with a simulated-to-real gap, strong lensing, is acknowledged as work in progress. The galaxy-property numbers (<1 dex for >80% of targets) are quoted without error bars or sample definitions; again, the details are in the cited papers.\n\nWho is this for? A reader who wants a five-page overview of what ULISSE can do, or a proceedings audience. As a research preprint it should be desk rejected; there is nothing here for a referee to check. That said, the paper is not misleading—it labels itself as a recap and gives credit to the source papers.\n\nMy recommendation: do not send this to peer review; it is a summary, not a research contribution, and the evidence that would need checking lives in the cited papers. For a proceedings volume that invites recaps, it is acceptable, but it does not deserve referee time.","headline":"A clearly written recap of the authors' own ULISSE work; no new results, and the core transfer claim remains under-defended, but it is an honest proceedings summary.","tokens_in":6413,"tokens_out":2398,"would_cite":false,"duration_ms":20229,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A single ImageNet-pretrained feature extractor, used with no labeled astronomy training, can support AGN detection, galaxy property estimation, strong-lensing search, and anomaly detection because distances in its feature space track…","keywords":["transfer learning","astronomical image analysis","feature extraction","EfficientNet","similarity search","AGN detection","anomaly detection","strong gravitational lensing"],"falsifier":"Run the same nearest-neighbor retrieval on a fixed galaxy sample with the ImageNet-pretrained EfficientNet features and again with the same architecture using random weights; if retrieval accuracy for AGN, stellar mass, or star formation rate is unchanged, then the ImageNet pretraining is not doing the work and the central transfer claim collapses.","tokens_in":5312,"feed_emoji":"🔭","tokens_out":10122,"duration_ms":82123,"temperature":0.7,"pith_summary":"The paper brings together four astronomical applications built on one idea: an ImageNet-pretrained EfficientNet, with its classification layer removed, converts any input image into a 1280-dimensional feature vector without any labeled astronomy training. Once every object is a point in this space, distances between points can be used to find similar objects, flag outliers, estimate physical properties, and detect rare events. The authors report that this label-free geometric representation identifies active galactic nuclei (AGN) candidates in three-band SDSS images, estimates galaxy stellar mass and star formation rate within about one dex for over 80% of targets, retrieves strong-lensing candidates with high accuracy in simulations, and flags bad epochs in time series. The point of the paper is that a single generic feature extractor can serve as a flexible foundation for many survey-analysis tasks, which matters as LSST-scale datasets make manually labeled training sets impractical.","feed_headline":"One ImageNet-trained network drives AGN, lens, and anomaly searches","feed_subtitle":"No labeled training data needed: a generic photo-trained network sorts galaxy images by physical similarity.","key_machinery":"The central object is ULISSE, a feature-extraction and similarity-search framework built on EfficientNet, a convolutional neural network pretrained on ImageNet with the classifier head removed. The network converts each input image into a 1280-dimensional feature vector, and the paper treats those vectors as points in a geometric space where distance is the similarity measure. That single machinery does the load-bearing work across applications: query-by-example retrieval for AGN and lens candidates, neighbor averaging for galaxy properties, and comparison against a stacked reference image for time-series anomalies and outlier detection.","core_discovery":"The central claim is that the visual features a convolutional network learns on ordinary photographs transfer to astronomy well enough that similarity in the resulting feature space tracks astrophysical similarity, not just visual appearance. Concretely, the paper transforms the problem of analyzing a complex astronomical image into a geometric problem in a high-dimensional feature space: each image becomes a point among 1280 features, and the distance between points defines which objects are most similar or dissimilar. On this basis the authors report that AGN candidates can be retrieved without training for the task, that stellar mass and star formation rate can be estimated from images alone by averaging over retrieved neighbors, that strong-lensing candidates stand out in simulated data with 94% accuracy in the first hundred most frequent candidates, and that outlier epochs in light curves can be flagged against a stacked-image reference. The claim is that this works despite the small, noisy, and heterogeneous character of astronomical survey data.","pith_inferences":["An implication the authors leave implicit is that if feature-space distance tracks physical similarity, the same extractor can index entire survey databases, letting astronomers query any new phenomenon by example rather than retraining a model.","The reported drop in strong-lensing performance from simulated to real data points to a boundary of the transfer: it likely works best when the query distribution matches the data distribution, and the effect of noise and depth changes on retrieval accuracy is a direct test of that boundary.","The stacked-image 'typical behaviour' criterion for time-series epochs could be turned into a real-time alert filter for survey streams, flagging only epochs that deviate beyond a dynamic threshold.","A direct comparison against self-supervised or astronomy-specific pretrained encoders would isolate how much of the success comes from ImageNet pretraining itself, an extension the authors gesture at when they mention improving the feature extractor."],"forward_implications":["AGN candidates can be retrieved from a single three-band image without any AGN-specific training, with reported retrieval efficiency above 33% for the tested prototype morphologies and up to 60% for composite objects.","Galaxy stellar mass and star formation rate can be estimated directly from images by averaging over retrieved neighbors, with over 80% of target objects within 1 dex of the estimate.","Strong-lensing candidates in simulated data appear at the top of the retrieval list: 94% of the first 100 most frequent candidates are true positives, and 93% of the first 200.","Problematic epochs in time series can be flagged automatically by comparing each epoch's feature vector with the stacked 'typical' image, without manual inspection.","The same feature space enables large-scale anomaly detection by defining normality as low distance to the nearest neighbor, offering a route to finding rare or unknown object classes in upcoming large surveys."],"supporting_citations":[{"why":"Introduces ULISSE, the feature-extraction and similarity-search framework that all four applications build on, and demonstrates the AGN detection baseline.","marker":"[10]"},{"why":"Supplies EfficientNet, the pretrained convolutional architecture whose first part becomes the feature extractor.","marker":"[11]"},{"why":"Provides ImageNet, the large dataset of ordinary photographs whose pretraining produces the transferred features.","marker":"[15]"},{"why":"Extends ULISSE to estimate stellar mass and star formation rate from images, the property-estimation application.","marker":"[16]"},{"why":"Provides the galSpec catalog of galaxy properties used as the reference sample for assessing the property estimates.","marker":"[17]"},{"why":"Applies the same feature extractor to identify problematic epochs in VST time series, the anomaly-detection application.","marker":"[20]"}],"fun_headline_variants":["ImageNet-trained CNN finds AGN, lens, and anomaly candidates","No labeled data needed: photo-trained net drives astronomy","Transfer learning: generic features solve astronomical tasks","One CNN, many insights: from AGN to lensing","Photo features map to physics: transfer learning in action"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the visual regularities a network learns from ordinary photographs also order astronomical images by astrophysical similarity, so that points close in the feature space are physically alike.","fun_headline_variants_meta":{"raw":{"variants":["ImageNet-trained CNN finds AGN, lens, and anomaly candidates","No labeled data needed: photo-trained net drives astronomy","Transfer learning: generic features solve astronomical tasks","One CNN, many insights: from AGN to lensing","Photo features map to physics: transfer learning in action"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000216,"raw_usage":{"total_tokens":1398,"prompt_tokens":874,"completion_tokens":524,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":490,"completion_tokens_details":{"reasoning_tokens":459}},"tokens_in":490,"tokens_out":524,"duration_ms":5276,"temperature":1.0,"reasoning_tokens":459,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T11:24:26.737845+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same nearest-neighbor retrieval on a fixed galaxy sample with the ImageNet-pretrained EfficientNet features and again with the same architecture using random weights; if retrieval accuracy for AGN, stellar mass, or star formation rate is unchanged, then the ImageNet pretraining is not doing the work and the central transfer claim collapses.","supporting_citations":[{"cited_title":"Doorenbos, O","cited_arxiv_id":null,"evidence_quote":"Introduces ULISSE, the feature-extraction and similarity-search framework that all four applications build on, and demonstrates the AGN detection baseline."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies EfficientNet, the pretrained convolutional architecture whose first part becomes the feature extractor."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides ImageNet, the large dataset of ordinary photographs whose pretraining produces the transferred features."},{"cited_title":"Torbaniuk, L","cited_arxiv_id":null,"evidence_quote":"Extends ULISSE to estimate stellar mass and star formation rate from images, the property-estimation application."},{"cited_title":"Brinchmann, S","cited_arxiv_id":null,"evidence_quote":"Provides the galSpec catalog of galaxy properties used as the reference sample for assessing the property estimates."},{"cited_title":"Cavuoti, D","cited_arxiv_id":null,"evidence_quote":"Applies the same feature extractor to identify problematic epochs in VST time series, the anomaly-detection application."}],"review_version":1}