{"id":"6e6af5ce-996f-4e8a-b8b6-5e4dabddcd27","arxiv_id":"2504.14426","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":10,"one_line_summary":"Halo formation times can be predicted from observable halo and galaxy properties with roughly 5 percent bias and 20 percent scatter using machine learning models trained on cosmological simulations.","lead":"This paper trains machine learning models to predict when a dark matter halo assembled half its mass, using properties of the halo and the galaxies inside it from cosmological simulations. The authors find that random forests and neural networks predict formation time with 4 to 9 percent bias and about 20 percent scatter, and simple linear formulas using a few observable galaxy properties work nearly as well.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Random halo-level train/test split across cluster regions leaks correlated information; reported biases and scatters may be optimistic without a cluster-grouped split.","rationale":"The reader's weakest assumption focuses on external validity: the sample is cluster-centric, drawn from one baryonic implementation (Gizmo-Simba), and incomplete at low mass, so the trained relations may not generalize to the field or other physics. That is a real limitation, and the authors themselves concede the Gizmo-Simba specificity in the conclusions. But it is a limitation of scope, not a threat to the reported internal accuracy. My concern is different and more load-bearing: the test-set metrics themselves may be unreliable because the random halo-level split leaks correlated information from shared cluster regions. The paper's own Figure 1 and feature importance analysis show that environment-sensitive features (com_offset, magnitude gaps) are the strongest predictors, which makes leakage plausible. If a cluster-grouped split reveals materially worse performance, the paper's headline numbers would need revision, not just its generalization claims. This concern is concrete, testable with the existing data, and directly targets the quantitative core of the paper, so it takes priority over the external-validity concern. The reader's verdict of CONDITIONAL remains appropriate, but the condition should include a cluster-split validation, which the reader did not explicitly request.","tokens_in":35305,"tokens_out":3646,"duration_ms":38025,"concrete_test":"Repeat the Model 6 RF and CNN Model 2 training using a cluster-grouped split: assign all halos from a random 15% of the 324 The300 cluster regions to the test set, with no region appearing in both training and test. Use the same hyperparameters, sample weights, and preprocessing as in the paper. Then compare the test-set median relative bias and the relative-error standard deviation with the values in Figures 3 (Model 6: 0.053 +/- 0.190) and 8 (CNN Model 2: 0.016 +/- 0.233). If the scatter increases by more than ~30% (e.g., above 0.25 for RF or above 0.30 for CNN), the reported accuracy is inflated by information leakage; if the metrics remain within ~10%, the concern is refuted.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central quantitative claim is the reported test-set accuracy: RF median biases of 4%-9% with ~20% scatter, and CNN biases below 4%. These numbers come from a single fixed 85/15 random split (Sec. 3.1) of 1,918 halos. However, those halos are not independent: they are drawn from only 324 zoom-in regions, each 15 h^-1 Mpc in radius (Sec. 2), so halos from the same cluster region share large-scale environment, tidal history, and often similar assembly histories. A random split places halos from the same region in both training and test sets, letting the models memorize region-specific signals. The features most important to the RF models (com_offset, M12, M14, MBCG/Msat) and the CNN radial maps are exactly the kind of environment-dependent quantities that could encode this leakage. The paper never reports a cluster-grouped split, nor does it quantify halo-to-cluster membership. Consequently, the headline error metrics may be optimistically biased; true generalization to an unseen cluster region could be substantially worse. This is an internal-validity concern that can be tested without any external simulation, making it more immediately decisive than the acknowledged Gizmo-Simba specificity.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper trains random forest (RF), convolutional neural network (CNN), and simple linear models to predict dark matter halo formation time t1/2 (the epoch when a halo has assembled half of its final mass) for 1,918 halos from the The300 Gizmo-Simba cluster zoom-in simulations. The RF models use six feature sets built from halo properties, BCG+ICL properties, and aperture-based baryonic properties; the CNNs take 2D radial maps of stellar and gas properties under three binning schemes; the linear models use the magnitude gaps M12 and M14 plus the mass ratio MBCG/Msat (Eqs. 3-5). Using a fixed 85/15 halo-level train/test split, the paper reports RF median relative biases of 4%-9% with ~20% scatter, CNN biases below 4% with somewhat larger scatter, and linear-model performance comparable to the RFs. The predictions are also shown to reproduce the M200c-t1/2 and cNFW-t1/2 trends and are compared with the Correa et al. (2015) EPS-based analytical relation.","tokens_in":35646,"tokens_out":6555,"duration_ms":58346,"significance":"If the reported accuracy is robust, the paper offers a practical, observationally motivated route to estimating halo formation time at cluster scales, and the simple linear relations in Eqs. 3-5 are directly usable by observers. The work is honest about its limitations: the results are explicitly stated to be specific to Gizmo-Simba physics (Section 7), the CNN saliency maps are acknowledged to be noisy and weakly correlated with t1/2 (Appendix B), and the ML pipeline is standard and well described. The main significance hinges on the credibility of the headline error metrics, which currently rest on a single halo-level split with no cluster-grouped validation and no quantitative baseline comparison; both issues are testable within the existing simulation suite and should be fixable in revision.","major_comments":[{"comment":"This is the most load-bearing issue because the central quantitative claims are the reported test-set accuracies, and the data structure makes the halo-independence assumption untenable.","section":"Sec. 2, Sec. 3.1, Figs. 3 and 8"},{"comment":"This is load-bearing because without a baseline the reported absolute scatter cannot be judged; the 'surpass' claim in the abstract is not quantitatively supported.","section":"Abstract, Sec. 5, Sec. 6, Eqs. 3-5"},{"comment":"This is a separate but related issue: even if the cluster-grouped split shows no leakage, the lack of error bars prevents any claim about which model is best.","section":"Sec. 3.1, Figs. 3, 8, and 12"}],"minor_comments":[{"comment":"The handling of NaN values in the property maps is unclear: bins with absent gas particles are said to be 'excluded' from the maps, but the maps are then 'resized later on to have the same dimensions'; please specify whether the excluded bins are masked, imputed, or dropped before the resize, since this affects the CNN input data.","section":"Sec. 4.1"},{"comment":"The half-stellar-mass radius Rhalf from the CAESAR catalog is replaced by the linear-fit proxy Rhalf,fit because of a scatter of approximately 8.78 kpc, but the scatter is not characterized or propagated; please state whether this replacement biases the Binning Method 3 maps and whether the CNN results are sensitive to this choice.","section":"Sec. 4.1"},{"comment":"The brute-force selection of MBCG/Msat and M14 as additional linear-model features is described only qualitatively; please list the full set of candidate features tested and the specific scoring metric used to select each additional term.","section":"Sec. 6"},{"comment":"The Conclusions quote biases of 7.3%-9.7% for the saliency-range RF models, but these results appear only in Appendix B.2 (Figure B.4); consider adding a one-line summary of these numbers in the main text so that the main-body conclusions are self-contained.","section":"Sec. 7 and Appendix B.2"},{"comment":"In the caption for the middle panel, 'as detailed in the first column of Table B.1' appears to be a typo; it should refer to the second column of Table B.1, since the panel corresponds to CNN Model 2.","section":"Figure B.4 caption"},{"comment":"The Pearson correlation between M200c and predicted t1/2 is reported as ~0.52 for RF and ~0.46 for CNN, higher than the true correlation of 0.39; this regression-to-the-mean effect deserves a brief comment, as it indicates the models are somewhat more mass-driven than the actual t1/2 values.","section":"Sec. 5"}],"recommendation":"major_revision","confidential_remarks":"The paper is within the scope of A&A and the ML methodology is standard and well described. I find no circularity: the target t1/2 is not used as a feature, and the linear coefficients are fit on the training set and evaluated on a held-out test set. The main concern is internal validation — the clustered nature of the zoom-in sample and the absence of a quantitative baseline — both of which are testable and fixable within the manuscript's scope. The authors should be encouraged to address these points rather than be deterred by them."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's the short version: the paper's real product is Eqs. 3-5, simple linear relations between t1/2 and magnitude gaps M12, M14, MBCG/Msat, with ~25% scatter. If those hold up, observers can get a cheap formation-time proxy from cluster photometry. The ML part is more of a means to that end, and its headline accuracy numbers should be taken with a grain of salt.\n\nWhat's new and good: The explicit fits are new; they confirm and quantify earlier magnitude-gap work. The systematic comparison of RF vs CNN on the same target with the same train/test split is fair. They show feature importance: com_offset and M12 are dominant, consistent with dynamical state and assembly history. They also check that predicted t1/2 reproduces the mass and concentration trends, which is a good sanity check. The paper is clear about being Gizmo-Simba-specific.\n\nSoft spots: The stress-test note is correct. The 1,918 halos live in 324 zoom-in regions, each 15 Mpc across. A random 85/15 split will put siblings in both train and test. With com_offset, M12, and radial maps as features, the models can memorize region-specific signals. The reported 4-9% biases and ~20% scatters are therefore optimistic until they do a cluster-grouped split. That's an internal-validity check they can run tomorrow, and it should be required. Also, there are no error bars on the metrics, no quantitative mass-only baseline (e.g., just using the M200c-t1/2 relation), and no code/data release. The post hoc exclusions of seven halos are documented, so that's minor.\n\nThe central claim--that baryonic features carry information beyond mass--is not falsified; the linear fits are simple enough to be robust. But the ML accuracies are not yet believable as generalization results.\n\nRecommended: send to peer review. The linear fits and the feature importance deserve scrutiny, and the split issue is addressable. If the authors add a group split and a mass-only comparison, this becomes a solid, citable paper for cluster observers.","headline":"A useful set of linear fits for halo formation time from magnitude gaps, with a real but fixable validation flaw in the ML part.","tokens_in":36160,"tokens_out":2216,"would_cite":true,"duration_ms":20592,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Galaxy observables, not merger trees, reveal when a dark matter halo formed.","keywords":["halo formation time","halo assembly history","machine learning","random forest","convolutional neural network","brightest cluster galaxy","intracluster light","magnitude gap"],"falsifier":"Take the linear model of Eq. (4), apply it to halos from a different hydrodynamical simulation with different feedback physics, or to field and group-mass halos in the same simulation, and compare the predicted t1/2 with that simulation's merger-tree half-mass times. If the median relative bias exceeds about 10% or the scatter grows well beyond 0.27, the claim of transferable prediction is falsified. A sharper test removes dynamically disturbed halos and checks whether M12 still predicts t1/2; if the predictive power largely vanishes, the apparent signal is mainly dynamical state rather than formation epoch.","tokens_in":35154,"feed_emoji":"🔭","tokens_out":5912,"duration_ms":53778,"temperature":0.7,"pith_summary":"This paper argues that the formation time of a dark matter halo—when it has acquired half of its current mass—can be predicted from observable galaxy properties rather than from its invisible merger history. Using cluster simulations, the authors train random forest, convolutional neural network, and simple linear models on halo, brightest-cluster-galaxy, and intracluster-light properties. Random forests recover formation time with median biases of 4% to 9% and scatter near 20%, while convolutional networks reduce median bias below 4% at the cost of slightly larger scatter. Strikingly, linear models using only two magnitude gaps and a BCG-to-satellite mass ratio perform about as well as the full random forest.","feed_headline":"Two galaxy gaps predict when a galaxy cluster formed","feed_subtitle":"Simple linear fits match full machine-learning accuracy on simulated halo formation times.","key_machinery":"The load-bearing object is t1/2 itself, defined through AHF merger trees as the epoch when the main progenitor has accreted half the halo's current mass. The predictive machinery has three tiers: random forests over the full set of halo, BCG, and intracluster-light features, with permutation importance identifying com_offset and the magnitude gaps M12 and M14 as dominant; convolutional networks trained on 6×40 radial property maps of stellar and gas mass, metallicity, age, and temperature, interpreted afterwards with saliency maps; and three explicit least-squares formulas in M12, M14, and MBCG/Msat that observers can apply directly.","core_discovery":"The central claim is that t1/2 is imprinted in baryonic observables, particularly the magnitude gaps between the brightest cluster galaxy and the second and fourth brightest substructures, the ratio of BCG stellar mass to satellite stellar mass, and the dynamical-state indicator com_offset. A random forest trained on all available halo and baryonic features reaches median relative errors between 4% and 9% with standard deviations of about 19% to 23%, while convolutional networks fed only six radially binned baryonic properties reach median biases of 0.4% to 4.1% with slightly larger scatter. Explicit linear relations—Eqs. 3, 4, and 5—give 3.5% to 5.3% bias and scatter of 0.25 to 0.27, matching random forest performance. The predicted formation times also reproduce the known correlations with halo mass and concentration.","pith_inferences":["The strong correlations of com_offset and M12 with t1/2 suggest the models may be reading dynamical state rather than assembly epoch per se; a control experiment that holds dynamical state fixed would isolate how much true assembly signal remains.","The linear calibration may be mass-dependent, so extending Eqs. 3-5 to group-mass halos below the cluster range would probably require refitting; this is testable on field or group simulations.","The CNN saliency ranges lying largely outside the BCG/ICL transition radius hint that outer stellar envelopes carry assembly information, which could be checked by measuring ICL colours and ages at roughly 60 to 100 kpc in real clusters.","Because only one baryonic feedback implementation was used, agreement of these relations with a different feedback model would be the strongest sign that the baryonic-to-assembly link is physical rather than simulation-specific."],"forward_implications":["If the linear relations hold beyond this training sample, observers can estimate cluster formation times from photometric magnitude gaps and BCG-to-satellite mass ratios alone, with no spectral or dynamical data.","The best random forest scatter of about 20% in t1/2 is enough to separate clusters into early- and late-forming samples for studies of environmental and assembly effects.","Reproducing the M200c-t1/2 and cNFW-t1/2 correlations with predicted values means the models can assign formation-time labels without tracing merger trees.","The low ranking of the ICL fraction in feature importance suggests surveys should prioritize magnitude gaps over intracluster-light fractions when targeting formation time."],"supporting_citations":[{"why":"Supplies the cluster zoom-in hydrodynamical simulations from which the halo and galaxy samples are drawn.","marker":"Cui et al. 2018"},{"why":"Defines the baryonic physics implementation used to produce the simulated galaxies in the training set.","marker":"Davé et al. 2019"},{"why":"Provides the AHF halo finder and merger-tree tool that define the halo catalog and the t1/2 target.","marker":"Knollmann & Knebe 2009"},{"why":"Establishes the half-mass formation-time definition that the paper adopts as its target quantity.","marker":"Wechsler et al. 2002"},{"why":"Introduces the magnitude-gap proxy M12 that anchors the linear models.","marker":"Golden-Marx & Miller 2018"},{"why":"Connects BCG and intracluster-light properties to halo formation time, motivating the baryonic feature set.","marker":"Golden-Marx et al. 2022"},{"why":"Provides the analytical extended Press-Schechter formation-time benchmark against which the machine learning predictions are compared.","marker":"Correa et al. 2015"},{"why":"Supplies the random forest regression algorithm used for the six halo-property models.","marker":"Breiman 2001"}],"fun_headline_variants":["Gaps in galaxy brightness date cluster assembly","Galaxy gaps alone rival machine learning for halo ages","Brightness gaps reveal halo formation epoch without complex models","Linear fits from galaxy gaps match ML for cluster ages","Two magnitude gaps decode cluster formation time"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the relation between baryonic observables such as magnitude gaps and BCG-to-satellite mass ratio and halo assembly time is universal enough to survive outside the specific cluster zoom-in sample and baryonic feedback implementation on which the models were trained.","fun_headline_variants_meta":{"raw":{"variants":["Gaps in galaxy brightness date cluster assembly","Galaxy gaps alone rival machine learning for halo ages","Brightness gaps reveal halo formation epoch without complex models","Linear fits from galaxy gaps match ML for cluster ages","Two magnitude gaps decode cluster formation time"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000531,"raw_usage":{"total_tokens":2583,"prompt_tokens":994,"completion_tokens":1589,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":610,"completion_tokens_details":{"reasoning_tokens":1518}},"tokens_in":610,"tokens_out":1589,"duration_ms":13456,"temperature":1.0,"reasoning_tokens":1518,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T11:48:50.074400+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take the linear model of Eq. (4), apply it to halos from a different hydrodynamical simulation with different feedback physics, or to field and group-mass halos in the same simulation, and compare the predicted t1/2 with that simulation's merger-tree half-mass times. If the median relative bias exceeds about 10% or the scatter grows well beyond 0.27, the claim of transferable prediction is falsified. A sharper test removes dynamically disturbed halos and checks whether M12 still predicts t1/2; if the predictive power largely vanishes, the apparent signal is mainly dynamical state rather than formation epoch.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the AHF halo finder and merger-tree tool that define the halo catalog and the t1/2 target."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Introduces the magnitude-gap proxy M12 that anchors the linear models."},{"cited_title":"B., Miller, C., Zhang, Y., et al","cited_arxiv_id":null,"evidence_quote":"Connects BCG and intracluster-light properties to halo formation time, motivating the baryonic feature set."}],"review_version":1}