{"id":"4c38efdd-6509-43e3-bfe9-ae2eb1b1a5ab","arxiv_id":"2411.19723","paper_version":3,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A random forest trained on TNG50 internal galaxy properties preclassifies lopsided versus symmetric disk galaxies with about 80% balanced accuracy, and similar accuracy is reached with photometric observables alone.","lead":"The authors trained a random forest on about 8,000 simulated disk galaxies to automatically label galaxies as lopsided or symmetric, using only internal properties like central density and tidal strength, and achieved roughly 80% balanced accuracy. The work offers a fast preselection tool for large photometric surveys, but the claim that lopsidedness is mainly internal relies on known correlations and lacks an environment-feature baseline.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Repeated snapshots of the same TNG50 galaxies enter both training and test sets, so the reported balanced accuracy (~0.81) may overstate generalization to unseen galaxies.","rationale":"Agree with the reader that the train/test split is the weak foundation. The paper is transparent about using multiple snapshots (Section 2.2), but this only makes the omission of grouping by galaxy ID more conspicuous: the same galaxy's evolved structure can appear on both sides of the split, so the reported 0.81 balanced accuracy likely mixes genuine generalization with individual-galaxy memorization. This is the load-bearing issue because the paper's practical claim is speed and accuracy on new survey galaxies; if the accuracy largely reflects repeated galaxies, that claim is unsupported. The missing environment-feature baseline is a real issue for the 'mainly internal' interpretation, but it does not undercut the classifier's internal-feature performance as directly as leakage does; moreover, the paper's own misclassification analysis (Sect. 4.3) already shows environmental interactions matter for a nontrivial fraction of cases, so the physical conclusion is appropriately qualified. The proposed grouped-split retest is cheap and definitive. If the drop is small, the paper's numbers stand; if large, the conditional verdict should be tightened.","tokens_in":20699,"tokens_out":4706,"duration_ms":41339,"concrete_test":"Re-train SMOTE+RF exactly as in Sections 3.2–4.1 (same hyperparameters from Table 2, same SMOTE strategy, same threshold 0.5) but replace the StratifiedShuffleSplit with a grouped split by unique TNG50 subhalo ID: assign all snapshots of a given galaxy to either the 70% training or 30% test partition, never both. Compare balanced accuracy, G-mean, ROC-AUC, and the confusion matrix against Tables 3 and 5 (and the observables-only model). If the grouped-split balanced accuracy drops by more than ~0.03–0.05, the reported generalization is inflated; if it remains within the quoted ±0.01 uncertainties, the leakage is not the dominant driver.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 2.2 states that galaxies are identified across z=0–0.5 and that a given galaxy 'will be present at different snapshots' and 'will serve as input for the training process.' Section 3.2 then splits the 7,919 rows into 70/30 with StratifiedShuffleSplit, without grouping by subhalo ID. Because a galaxy's structural features (µ*, R50, Rext, M50, SFR) and its A1 label are strongly autocorrelated across lookback times of ≤ ~5 Gyr, the test set can contain later snapshots of galaxies whose earlier snapshots were used in training. The classifier can therefore exploit per-galaxy identity rather than learning a general internal-property rule. This makes the headline accuracies in Tables 3 and 5 (balanced accuracy 0.813±0.010 and 0.799±0.009) estimates of performance on partially seen galaxies, not on independent galaxies. The central practical claim—a fast preselector for survey galaxies—requires generalization to unseen systems, which this split does not establish.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper trains random forest classifiers on 7,919 late-type galaxy snapshots from TNG50, labeling each snapshot as lopsided or symmetric using the radial average of the m=1 Fourier amplitude A1 over the interval R50–1.4R90 with a threshold of 0.1. The authors compare two imbalance-handling approaches, SMOTE+RF and Balanced Random Forest, and report a balanced accuracy of 0.813±0.010 (Table 3), with similar performance (~0.799±0.009, Table 5) when using photometrically observable features (R50, Rext, c/a, SFR, L50). They interpret the high accuracy as evidence that lopsidedness is mainly a tracer of internal galaxy structure, and propose the classifier as a fast preselection tool for large multiband surveys. The paper also includes a feature-importance analysis, a study of misclassified galaxies with illustrative snapshots and interaction histories, and a discussion of the physical meaning of borderline A1 classifications.","tokens_in":20899,"tokens_out":8751,"duration_ms":71088,"significance":"If the reported accuracy reflects generalization to unseen galaxies, this would be a useful practical contribution: a fast, automated preselector for lopsided galaxies, with a clearly described pipeline and careful treatment of class imbalance (SMOTE and BRF, balanced accuracy as primary metric). The case studies of misclassified galaxies in Section 4.3 are informative and provide physical insight into the borderline nature of the A1-based label. However, the two concerns below—train/test leakage from repeated snapshots of the same galaxies and the self-predictive relationship between features and label—mean that the headline accuracies cannot currently be taken at face value, and the physical interpretation in Section 5 is stronger than the evidence supports. The analysis is reproducible in principle, though no code is provided.","major_comments":[{"comment":"The sample consists of galaxy snapshots over z=0–0.5; Section 2.2 explicitly states that a given galaxy 'will be present at different snapshots of the simulation' and 'will serve as input for the training process.' The train/test split in Section 3.2 uses StratifiedShuffleSplit on the 7,919 rows without grouping by subhalo ID. Because the structural features (µ*, TP, R50, Rext, M50, SFR) and the A1 label are strongly autocorrelated across lookback times up to ~5 Gyr, later snapshots of a galaxy used in training can appear in the test set, allowing the random forest to exploit per-galaxy identity rather than a general rule. The balanced accuracies in Tables 3 and 5 are therefore estimates of performance on partially seen galaxies, not on independent galaxies, and do not yet support the survey preselection claim. The authors should redo the split at the galaxy level (e.g., GroupShuffleSplit or GroupKFold on subhalo ID) and report metrics for galaxies whose entire evolutionary tracks were excluded from training.","section":"Sections 2.2 and 3.2; Tables 3 and 5"},{"comment":"The label A1 is the radially averaged m=1 Fourier amplitude of the stellar mass distribution over R50–1.4R90, while the top-ranked features in Table 4—µ*, TP, M50, and also R50 and Rext—are integrals or moments of the same simulated stellar mass distribution. The classifier is thus mapping one set of summary statistics of a density field onto another summary statistic of the same field. The high accuracy therefore does not by itself test whether lopsidedness is 'mainly a tracer of galaxies internal structures' rather than of environment, since no environment-based features are included for comparison. The authors should either temper this interpretation (noting that the result is partly a self-consistency check of the mass distribution), or add a control experiment that includes environment features (e.g., host halo mass, local density, distance to nearest neighbor) and compares accuracies or feature importances, to directly address the environment hypothesis.","section":"Sections 3.1, 2.2 (Table 1), and 5"}],"minor_comments":[{"comment":"The abstract states '≈ 8000 late-type galaxies' while Section 2.2 gives 7,919; please harmonize the numbers.","section":"Abstract and Section 2.2"},{"comment":"The BRF TNR row reports the uncertainty as 0.00; if this is a rounding artifact, please give at least two significant digits.","section":"Table 3"},{"comment":"The text states that the observational-feature model produces 535 misclassifications, 'a 15% increase' relative to the 455 misclassifications of the full-feature model; the increase is actually 17.6%, and the wording should clarify that this is an increase in errors.","section":"Section 4.4"},{"comment":"Permutation importance is computed on the test set only; this is nonstandard because the test set is meant to be used once. Consider computing permutation importance on a validation split or on the training set to avoid optimistic feature-importance estimates.","section":"Section 4.2"},{"comment":"Minor typographical issues include 'stellar participles' in Section 3.1 and 'Richter & Sacisi 1994' in the references (should be 'Sancisi'); also, the numbers quoted for correct/incorrect classifications in Fig. 5 are slightly inconsistent with the reported TPR/TNR values, so the rounding should be checked.","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":"The paper is well-written and the topic is well within the scope of A&A. The leakage issue is serious but fixable within the manuscript's scope: a galaxy-level split and a revised interpretation of the environment claim would make the paper sound. If the authors cannot obtain group-level splits, the claims should be substantially downgraded. I do not see grounds for rejection if the leakage is addressed."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know about this paper if you care about automated morphological classification for upcoming photometric surveys. The authors train a random forest on ~8,000 TNG50 disk galaxies to pre-classify lopsided vs symmetric galaxies using internal structural parameters, then show that a version using only photometric observables plus SFR reaches nearly the same balanced accuracy (~0.80). That photometric-only result is the genuinely practical contribution; nobody has shipped that as a preselection tool before. The paper also does careful work on the misclassified cases, using orbital histories to show that about a quarter of the \"false lopsided\" cases had recent strong interactions. That section is worth reading.\n\nThe soft spots are real, and the stress-test note is correct. The main issue is the train/test split. The authors use stratified shuffle split on individual galaxy snapshots across z=0–0.5, and they explicitly note that the same galaxy appears at multiple snapshots. Nothing groups rows by galaxy ID. So a galaxy's evolved snapshot can land in the test set while an earlier or later snapshot sits in the training set. The ~0.81 balanced accuracy is therefore an estimate of performance on partially seen galaxies, not on independent systems. That does not kill the tool, but it means the headline numbers are optimistic, and the paper should either redo the split grouping by Subfind ID or validate on TNG100 or observed data.\n\nThe second weakness is the physical conclusion. The paper argues that because the classifier uses only internal properties, the strong performance supports the hypothesis that lopsidedness is mainly a tracer of internal structures. That inference is too strong. Internal properties correlate with environment, and the paper never trains a baseline that includes environmental features (host halo mass, local density) to show they add nothing. The circularity concern is milder: the top features (μ*, TP) are integrals of the same stellar mass distribution used to compute A1, so part of the accuracy is by construction. But that is not fatal; the photometric version also works, and the features are not literally the same as the m=1 amplitude.\n\nOverall, this is a competent paper with a useful practical result and a fixable methodological gap. I would send it to peer review and ask for the ID-grouped split and an environment baseline before accepting the central claim.","headline":"A useful random-forest preselection tool for lopsided galaxies, but the accuracy numbers are inflated by snapshot leakage and the physical conclusion overreaches without an environment baseline.","tokens_in":21436,"tokens_out":2877,"would_cite":false,"duration_ms":26911,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A random forest trained only on internal galaxy properties can pre-classify lopsided versus symmetric disk galaxies with 81% balanced accuracy, supporting the view that lopsidedness is mainly a tracer of internal structure.","keywords":["lopsided galaxies","galaxy classification","random forest","TNG50 simulation","Fourier decomposition","galaxy internal structure","machine learning","photometric surveys"],"falsifier":"Retrain the identical SMOTE+RF pipeline using a galaxy-grouped split that keeps all snapshots of the same galaxy in the same fold, and measure balanced accuracy on held-out galaxies. If the grouped-split accuracy drops well below 0.81 (toward 0.5–0.6), the reported generalization is inflated by snapshot overlap. A complementary check is to run the trained classifier on an independent observed sample of disk galaxies whose A1 values are measured from imaging and to compare predicted versus actual labels.","tokens_in":20489,"feed_emoji":"🌌","tokens_out":9146,"duration_ms":74288,"temperature":0.7,"pith_summary":"Lopsided galaxies, whose stellar disks are asymmetrically distributed, are common but hard to characterize at survey scale. The paper claims that a random forest trained only on internal galaxy properties can quickly pre-classify simulated disk galaxies as lopsided or symmetric, with balanced accuracy 0.813 ± 0.010 on the TNG50 test set and 0.799 ± 0.009 when only photometrically observable features are used. Because no environmental information enters the features, the authors read the high accuracy as strong support for the hypothesis that lopsidedness is mainly a tracer of a galaxy's internal structure. The practical payoff would be a fast preselection tool for large multiband photometric surveys, with Fourier decomposition reserved for the smaller pre-selected sample.","feed_headline":"Random forest flags lopsided galaxies at 81% accuracy","feed_subtitle":"No environment data needed: galaxy structure alone predicts the lopsided label, even with photometric-only features.","key_machinery":"The load-bearing object is A1, the radially averaged amplitude of the m=1 mode of the stellar mass surface density, computed within a cylinder of width 1.4R90 and height 2h90 and averaged over the radial interval R50–1.4R90; galaxies with A1>0.1 are labeled lopsided. This Fourier label is the ground truth. The classifier is a random forest ensemble, with SMOTE oversampling used to balance the minority symmetric class, trained on ten internal parameters; permutation importance shows that central stellar mass density μ*, tidal parameter TP, and star formation rate carry most of the predictive signal, with μ* dominant. In the observable-feature variant, M50 is replaced by r-band luminosity within R50 and only R50, Rext, c/a, and SFR remain as inputs, while the training pipeline stays the same.","core_discovery":"The paper's central claim is that the m=1 Fourier asymmetry A1, averaged over the radial interval R50–1.4R90 and thresholded at 0.1, can be predicted from a small set of internal galaxy parameters. On a held-out 30% of the TNG50 sample, the SMOTE+RF classifier reaches balanced accuracy 0.813±0.010, and substituting r-band luminosity and other photometric proxies for the mass-based features leaves the performance almost unchanged at 0.799±0.009. The authors interpret this as evidence that lopsidedness is primarily an internal-structure phenomenon, not a direct map of present-day environment. They also find that the model's errors cluster near the A1=0.1 threshold and are physically intelligible: some misclassifications have symmetric interiors with a recently tidally disturbed outer disk, while others have lopsided-prone interiors that are merely unperturbed at the current snapshot.","pith_inferences":["Because the paper's split is over snapshots, not over unique galaxies, and each TNG50 galaxy appears at several redshifts, a galaxy-grouped cross-validation is the natural next test; I would expect the true generalization accuracy for never-seen galaxies to be lower than the reported 0.813.","The authors' own misclassification analysis suggests lopsidedness is partly a transient state: a galaxy can be structurally prone to lopsidedness but currently symmetric, or internally symmetric but temporarily lopsided after an interaction. Pushing this further, a single snapshot label mixes at least three physically distinct populations, so evolutionary conclusions drawn from pre-selected sample","A testable extension would be to train the same pipeline separately on the three subsets defined by the misclassification analysis (recently perturbed, structurally prone, borderline) and check whether their feature distributions are separable; if they are, a three-class or regression formulation would be more informative than a binary preselector.","For real photometric surveys, the simulation-trained model will face redshift-dependent surface brightness dimming, PSF dilution, and differences between simulated and observed SFR proxies; re-calibrating on a labeled observed subsample would be necessary before the reported ~80% accuracy is trusted on survey data."],"forward_implications":["Large multiband surveys can use the classifier as a fast preselection step, flagging lopsided candidates and limiting expensive Fourier decomposition or visual inspection to a smaller subset.","The high accuracy with no environment features supports treating lopsidedness as a structural indicator, which simplifies theoretical interpretation of survey samples.","The observable-feature version at ~80% balanced accuracy suggests the method transfers to photometric catalogs without stellar population or dynamical modeling of each galaxy.","The misclassification structure implies the pre-selected sample will still be clean enough for statistical studies as long as borderline A1 cases are treated as a separate, low-confidence category."],"supporting_citations":[{"why":"Supplies the m=1 Fourier amplitude method for measuring lopsidedness and the observational ~30% incidence used as context.","marker":"Zaritsky & Rix (1997)"},{"why":"Shows observationally that lopsided galaxies have lower central concentration and stellar mass density, the correlation the classifier exploits.","marker":"Reichard et al. (2008)"},{"why":"Finds in TNG50 that lopsidedness anticorrelates with the tidal parameter TP, motivating the internal-feature approach.","marker":"Varela-Lavin et al. (2023)"},{"why":"Provides the galaxy selection criteria and the TNG50 finding that lopsided and symmetric galaxies have different assembly histories.","marker":"Dolfi et al. (2023)"},{"why":"Reviews disk self-gravity and internal dynamical mechanisms for lopsidedness, used to interpret feature importance and misclassifications.","marker":"Jog & Combes (2009)"},{"why":"Introduces the random forest ensemble method on which both classifiers are built.","marker":"Breiman (2001)"},{"why":"Introduces Balanced Random Forests, the imbalanced-data baseline compared against SMOTE+RF.","marker":"Chen & Breiman (2004)"},{"why":"Introduces SMOTE oversampling, the method used to balance the minority symmetric class in the selected classifier.","marker":"Bowyer et al. (2011)"}],"fun_headline_variants":["AI spots lopsided galaxies from internal structure alone","Random forest boasts 81% accuracy in lopsided galaxy hunt","No environment needed: AI identifies lopsided galaxies","Machine learning reveals lopsided galaxies via photometry","Galaxy asymmetry predicted by internal features with ML"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The key load-bearing assumption is that randomly splitting the simulated galaxy snapshots into training and test sets gives independent samples, even though each galaxy reappears at multiple snapshots between z=0 and z=0.5; if later snapshots of a galaxy resemble the earlier ones used for training, the ~81% balanced accuracy could partly reflect memorization of individual systems rather than generalization to unseen galaxies.","fun_headline_variants_meta":{"raw":{"variants":["AI spots lopsided galaxies from internal structure alone","Random forest boasts 81% accuracy in lopsided galaxy hunt","No environment needed: AI identifies lopsided galaxies","Machine learning reveals lopsided galaxies via photometry","Galaxy asymmetry predicted by internal features with ML"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000471,"raw_usage":{"total_tokens":2390,"prompt_tokens":1037,"completion_tokens":1353,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":653,"completion_tokens_details":{"reasoning_tokens":1275}},"tokens_in":653,"tokens_out":1353,"duration_ms":8090,"temperature":1.0,"reasoning_tokens":1275,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T05:53:12.769642+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Retrain the identical SMOTE+RF pipeline using a galaxy-grouped split that keeps all snapshots of the same galaxy in the same fold, and measure balanced accuracy on held-out galaxies. If the grouped-split accuracy drops well below 0.81 (toward 0.5–0.6), the reported generalization is inflated by snapshot overlap. A complementary check is to run the trained classifier on an independent observed sample of disk galaxies whose A1 values are measured from imaging and to compare predicted versus actual labels.","supporting_citations":[{"cited_title":"A., Heckman, T","cited_arxiv_id":null,"evidence_quote":"Shows observationally that lopsided galaxies have lower central concentration and stellar mass density, the correlation the classifier exploits."},{"cited_title":"A., Tissera, P","cited_arxiv_id":null,"evidence_quote":"Finds in TNG50 that lopsidedness anticorrelates with the tidal parameter TP, motivating the internal-feature approach."},{"cited_title":"A., Monachesi, A., et al","cited_arxiv_id":null,"evidence_quote":"Provides the galaxy selection criteria and the TNG50 finding that lopsided and symmetric galaxies have different assembly histories."},{"cited_title":"& Breiman, L","cited_arxiv_id":null,"evidence_quote":"Introduces Balanced Random Forests, the imbalanced-data baseline compared against SMOTE+RF."}],"review_version":1}