{"id":"282039d3-33d6-4364-b1b3-eaf1b62bd09d","arxiv_id":"2411.13321","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"A regression model using WISE fluxes and quality flags predicts Spitzer MIPS1 and IRAC4 fluxes with R² ~0.94-0.98, recovering faint sources where WISE overestimates.","lead":"The authors trained machine learning models on paired WISE and Spitzer observations of open cluster stars to estimate Spitzer-quality 8 and 24 micron fluxes from WISE data alone. This could let astronomers use the all-sky WISE catalog with greater confidence in faint mid-infrared measurements, which currently suffer from source confusion.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Central claim is supported on the labeled population; the load-bearing gap is that accuracy on sources without Spitzer detections is assumed, not demonstrated.","rationale":"The reader's weakest_assumption correctly identifies cross-population transfer from open cluster members to the broader allWISE population as the least secure step. The reported R^2 and MAPE are computed on a random split of sources with Spitzer detections, so they establish that the ET model can reproduce Spitzer fluxes for sources in the same selection regime. The application to sources without Spitzer detections is the defining use case of the paper, and it is supported only by (i) the physical assumption that the WISE-Spitzer relation is stable across SED types and (ii) a two-star IC 348 demonstration. I do not find an internal inconsistency in the metric computation, and the feature-selection-before-split protocol is a secondary methodological concern rather than the primary obstacle. The central quantitative claim about the labeled test population is credible; the generalization claim is real but unproven. A conditional verdict with a concrete external validation requirement is therefore appropriate, and the reader's conditional verdict already reflects this. I would not change the verdict, but the concrete test above should be run before the method is adopted for new samples.","tokens_in":31931,"tokens_out":7716,"duration_ms":94692,"concrete_test":"Build an external validation set from SEIP: all sources with W4>0, cc_flags='0000', ext_flg=0, and MIPS1 detections that are NOT in the HR24 open-cluster sample. Stratify by W4 flux (below vs above ~10^3 microJy) and by W1-W3 color (photosphere-like vs IR-excess candidates). Apply the trained ET model and compute per-stratum R^2 and median offset between predicted and observed MIPS1. If the low-W4, IR-excess stratum shows a systematic underestimate larger than roughly twice the reported MAPE, the claim that the model transfers to other allWISE samples fails.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's evidence for R^2~0.94 (MIPS1) and R^2~0.98 (IRAC4) comes from train/test splits of the labeled subset, i.e., sources that have Spitzer detections. The intended application, however, is to predict fluxes for the ~80% of clean sources that lack MIPS1 measurements (Section 2, Fig. 2). For those sources the learned WISE-to-Spitzer mapping is assumed to hold, but no held-out validation exists in the no-detection regime. The only external check is the IC 348 SED comparison (Section 4.2), which uses two hand-selected stars that do have Spitzer detections and does not probe the population without Spitzer data. If the W4-to-MIPS1 relationship depends on SED type, for example for sources with mid-IR excess where a bright W4 flux is real rather than contamination, a model trained on open cluster members could systematically suppress genuine 24 micron excesses. The convex-hull filter on W1-W4 (Section 2) ensures unlabeled sources lie inside the training envelope in WISE feature space, but it does not guarantee that the conditional distribution of the Spitzer target given those WISE features is the same as in the labeled set.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper trains machine-learning regression models, specifically extremely randomized trees (ET), on a sample of open cluster members cross-matched between the SEIP Spitzer catalog and allWISE to predict Spitzer MIPS1 (24 um) and IRAC4 (8 um) fluxes from WISE W1-W4 fluxes and quality flags. The authors report coefficients of determination R2~0.94 for MIPS1 and R2~0.98 for IRAC4, compare ET to a linear baseline, perform feature selection with Boruta, and construct prediction uncertainties from the spread of tree predictions. They apply the trained models to sources without Spitzer detections and illustrate the results on spectral energy distributions of two members of IC 348. The claimed contribution is a method to obtain Spitzer-equivalent mid-infrared photometry over the WISE sky, particularly for faint sources where WISE and Spitzer disagree most.","tokens_in":32207,"tokens_out":4555,"duration_ms":50240,"significance":"If the method's accuracy extends to the unlabeled population, it would be a practically valuable tool: it would let investigators use the all-sky WISE catalog with photometry calibrated to Spitzer's higher resolution and sensitivity, and it explicitly targets the faint-flux regime where raw WISE photometry is most discrepant. The paper is also commendable for comparing against a linear baseline, reporting feature importances, and showing learning curves with training-set size. However, the significance is currently bounded by validation gaps: the headline metrics are cross-validation numbers on labeled sources, and the only external test is a two-star, hand-selected SED comparison in IC 348, so the method's usefulness for the intended no-detection application is asserted rather than demonstrated.","major_comments":[{"comment":"The paper quotes R2~0.94 and R2~0.98 as the headline results, but the text states that the metrics in Table 2 'correspond to the average metrics obtained during the 10-fold cross-validation step,' not to predictions on the independently held-out test set. Section 3.2.1 reserves the test set for independent assessment and Fig. 5 shows test-set scatter, yet no test-set R2, MAPE, MAE, or RMSE are reported. Please report the test-set metrics explicitly and clarify whether the quoted R2 values are CV estimates or held-out test-set estimates; this distinction is central to the claim of predictive performance on sources not seen in training.","section":"Sec. 4.1, Table 2"},{"comment":"Feature selection with Boruta is applied to the full labeled dataset before the train/test split. The workflow in Fig. 3 shows that the labeled dataset is used to select important features and that only afterward is the 'remaining training data' split into train and test sets. This leaks information from the test set into the feature choice, which can bias cross-validated performance estimates optimistically. Feature selection should be nested inside the cross-validation procedure, or at least performed on the training folds only, before the reported metrics are accepted as unbiased.","section":"Sec. 3.2.1 and Fig. 3"},{"comment":"The 'real-world' validation on IC 348 consists of two hand-selected stars, both of which have Spitzer detections and were chosen after an exploratory SED examination. This does not validate the main intended application: predicting fluxes for the roughly 80% of clean sources that lack MIPS1 measurements (Fig. 2). The convex-hull filter described in Sec. 2 ensures that unlabeled sources lie inside the WISE-feature envelope of the training set, but it does not guarantee that the conditional distribution of Spitzer flux given WISE features is the same for labeled and unlabeled sources. Please provide a quantitative check in the no-detection regime, for example by holding out a random subset of labeled sources and treating them as unlabeled, or by comparing predictions with Spitzer upper limits or an independent catalog.","section":"Sec. 4.2 and Fig. 2"},{"comment":"The uncertainty intervals are computed as three times the standard error of the mean of the individual tree predictions. Because trees in an ExtraTrees ensemble are not independent draws from a posterior distribution and the ensemble spread does not include irreducible noise, these intervals are likely to undercover the true flux uncertainty. For a method intended to identify infrared excesses, where error bars determine whether an excess is significant, a coverage calibration on the test set (e.g., the fraction of true fluxes within the reported 3-sigma intervals) should be reported or the uncertainty model should be revised.","section":"Sec. 3.1.3"}],"minor_comments":[{"comment":"There are several typographical errors: 'mid-infared' in the abstract, 'proofed reliable data' in Sec. 4.2, 'loosing data' in Sec. 5, and 'riveting solution' in Sec. 5 should be 'losing data' and 'attractive solution' (or similar).","section":"Abstract and Sec. 4.2"},{"comment":"The explanation of the count mismatch (e.g., 1096 sources with MIPS1 plus 4348 without does not sum to 9231) appears only in the body text; adding a sentence to the Fig. 2 caption about the convex-hull filtering step would make the figure self-explanatory.","section":"Sec. 2, Fig. 2"},{"comment":"The description of the Boruta statistical test says features 'with probabilities below the confidence level alpha' are deemed irrelevant, which is the reverse of the usual Boruta decision rule; please clarify whether the intended criterion is that the feature importance is not significantly larger than the shadow features.","section":"Sec. 3.3.2"},{"comment":"The text says the Table 2 metrics are 'a pessimistic estimation' because the final model is retrained on train+test; this would be more convincing if the test-set metrics were shown alongside, particularly since the test-set scatter in Fig. 5 appears consistent with the CV values but no quantitative comparison is provided.","section":"Sec. 4.1"}],"recommendation":"major_revision","confidential_remarks":"The core idea is sensible and the paper is generally clearly written, but the validation of the central application is currently insufficient to support the advertised generality. The headline R2 values need to be clarified as CV versus test-set metrics, and the feature-selection leakage should be fixed with a proper nested procedure. The IC 348 demonstration, while illustrative, does not validate the no-detection regime; a quantitative holdout-style validation on labeled sources treated as unlabeled would substantially strengthen the paper. I do not see a fundamental flaw that would require rejection, but the load-bearing validation points need work before the paper can be accepted."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this paper does what it says for sources with Spitzer detections, but the application that matters—predicting 24 µm fluxes for the ~90% of clean WISE sources without MIPS1 detections—is never validated. That is the load-bearing gap. Everything else is mostly sound.\n\nWhat's new: applying ExtraTrees + Boruta to map WISE W1–W4 and quality flags to Spitzer MIPS1/IRAC4 for open cluster members is a new and sensible use of standard ML. The training/test protocol is proper: 10-fold CV, 80/20 split, learning curves, and a linear regression baseline. R² ~0.94/0.98 with MAPE below 3.5% on held-out test sets are believable, and the plots show predictions tracking the diagonal down to faint fluxes. The authors are appropriately careful in places: they define \"predict\" as statistical prediction and note a small systematic bias, recommending statistical use rather than individual fluxes.\n\nThe soft spots: (1) The external check in IC 348 uses two hand-selected stars, both with Spitzer detections; it demonstrates concordance but does not test the no-detection regime. The stress-test concern is on target: a model trained on open cluster members can learn a W4–MIPS1 relation that suppresses genuine 24 µm excesses, which are exactly what disk searchers care about. The convex-hull filter keeps sources inside the training envelope in WISE space, but that does not guarantee the conditional distribution of MIPS1 given WISE is unchanged. (2) The uncertainty intervals assume tree independence in the standard error of the mean; ET trees are not independent, so the error bars are somewhat optimistic. Minor, and they do flag it as a simplification. (3) No code or data release, so the result is not independently reproducible without contacting the authors.\n\nWho it's for: researchers wanting Spitzer-quality 8/24 µm photometry for large WISE samples, especially in open clusters and star-forming regions. It deserves a serious referee, with a request to address the unlabeled-regime validation—for example, testing on sources with Spitzer upper limits or pulling a small archival sample with actual detections.\n\nRecommendation: send it out. The central claim is supported on the labeled population, the gap is well-defined, and the fix is straightforward.","headline":"Solid held-out mapping of WISE to Spitzer fluxes, but the paper's real selling point—predicting Spitzer fluxes for sources without Spitzer detections—is never validated in that regime.","tokens_in":32779,"tokens_out":2114,"would_cite":true,"duration_ms":24693,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that extremely randomized trees can predict Spitzer MIPS1 24 µm and IRAC4 8 µm fluxes from WISE fluxes and quality flags with $R^2 \\approx 0.94$ and $R^2 \\approx 0.98$, providing Spitzer-equivalent photometry across the…","keywords":["machine learning","extremely randomized trees","WISE","Spitzer","mid-infrared photometry","spectral energy distributions","open clusters","infrared excess"],"falsifier":"Take a set of allWISE sources that do have SEIP Spitzer detections but were not used in training because they are not open-cluster members or fail the clean-sample flags, apply the trained model, and compare predicted against measured 8 µm and 24 µm fluxes; if $R^2$ falls well below the reported 0.98 and 0.94 values, or if a systematic bias appears at fluxes below about $10^3$ µJy, the generalization claim fails.","tokens_in":31735,"feed_emoji":"🔭","tokens_out":6602,"duration_ms":65782,"temperature":0.7,"pith_summary":"The paper tries to establish that the all-sky WISE catalog can be upgraded to Spitzer-like mid-infrared photometry without new observations: a machine-learning regressor trained on paired WISE and Spitzer measurements predicts 24 µm (MIPS1) and 8 µm (IRAC4) fluxes from WISE fluxes and quality flags. If correct, this matters because Spitzer's pointed mission has ended while WISE covers the entire sky, and raw WISE fluxes at faint levels are contaminated and disagree with Spitzer by up to an order of magnitude. The authors show that extremely randomized trees reach coefficients of determination of about 0.94 at 24 µm and 0.98 at 8 µm, and that the largest accuracy gain is exactly at the faintest fluxes, where the two surveys previously diverged. They demonstrate concordance on spectral energy distributions of IC 348 members and argue the approach generalizes to other allWISE samples with SEIP positional counterparts.","feed_headline":"ML predicts Spitzer-quality 8 and 24 µm fluxes from WISE data","feed_subtitle":"Models reach R² of 0.98 and 0.94, extending Spitzer-quality photometry across the WISE all-sky catalog.","key_machinery":"The central object is the extremely randomized trees (ET) regressor, an ensemble of decision trees in which both the feature subset and the split value at each node are chosen at random, with the final prediction obtained by averaging over trees. The pipeline first log-transforms the Spitzer target fluxes, applies Yeo-Johnson power transforms and standardization to the WISE predictors, selects relevant features with the Boruta algorithm, tunes hyperparameters through 10-fold cross-validation and randomized search, and estimates each prediction's 3σ uncertainty from the standard error of the individual tree predictions. This machinery carries the argument because it is what converts the noisy, partly contaminated WISE features into Spitzer-equivalent fluxes with small, well-characterized uncertainty intervals.","core_discovery":"The central discovery is that a regression function learned from WISE variables can reproduce Spitzer mid-infrared fluxes closely enough to replace them for sources never observed by Spitzer. Using a training set of high-quality open cluster members with paired allWISE and SEIP data, extremely randomized trees predict MIPS1 24 µm fluxes with $R^2 \\approx 0.94$ and IRAC4 8 µm fluxes with $R^2 \\approx 0.98$, with mean absolute percentage errors below about 3.5%. The predicted fluxes stay close to the one-to-one line across the whole flux range, including the faint regime below roughly $10^3$ µJy where raw WISE W3 and W4 fluxes show the largest overestimations and scatter. The authors interpret this as evidence that the model captures hidden relationships between the two surveys, and that the approach can supply Spitzer-quality photometry for WISE-only sources while avoiding the massive data loss that stricter WISE quality cuts would incur.","pith_inferences":["I would expect the trained model to transfer less cleanly to source populations absent from the open-cluster training set, such as heavily embedded young stellar objects or dusty galaxies; a direct validation on non-cluster SEIP sources with real Spitzer measurements would quantify the drop.","The same pipeline could be applied to other overlapping survey pairs, for example predicting AKARI or IRAS mid-infrared fluxes from WISE, wherever the relationship between bandpasses is smooth enough for a regression model to learn.","Because the paper reports a slight systematic overestimation below roughly $3\\times 10^3$ µJy at 24 µm, users should treat individual faint predictions as statistically useful rather than exact fluxes, and should propagate the provided uncertainties when classifying disk-excess candidates.","The feature-importance analysis suggests the 24 µm model relies on W4 at bright fluxes and switches to W3, W1, W2, and quality flags at faint fluxes; locating that switch threshold precisely could clarify the physical cause of the WISE-Spitzer discrepancy."],"forward_implications":["For the clean-sample open cluster members without Spitzer detections, the model produces 24 µm fluxes for roughly 4,348 sources instead of discarding them, multiplying the usable sample about fourfold.","At the faint end, where W4 and MIPS1 deviate by up to an order of magnitude, the ET predictions remain close to the diagonal, removing most of the scatter that plagues raw WISE photometry below about $10^3$ µJy.","Because the learning curves show $R^2$ and MAPE improving monotonically with training-set size, retraining on larger future catalogs is expected to improve the predictions further.","The ET models supply per-source 3σ uncertainty intervals from the spread across trees, with most relative errors below 10%, so the predicted fluxes can stand in for Spitzer data in SED fitting and infrared-excess searches.","The approach avoids the data loss of conventional quality cuts: Appendix A shows that a signal-to-noise based selection would retain only about 508 to 650 useful sources, compared with thousands covered by the predicted fluxes."],"supporting_citations":[{"why":"Supplies the census of 3530 open clusters with Gaia DR3 membership probabilities that defines the HR24 training sample.","marker":"Hunt & Reffert (2024)"},{"why":"Documents the allWISE release, its fluxes, and the quality flags used both as predictors and as sample-cleaning criteria.","marker":"Cutri et al. (2013)"},{"why":"Describes the WISE mission, its four bands, sensitivity, and all-sky coverage that motivate replacing raw WISE photometry.","marker":"Wright et al. (2010)"},{"why":"Describes the Spitzer mission and its IRAC and MIPS instruments, defining the target fluxes the model tries to predict.","marker":"Werner et al. (2004)"},{"why":"Introduces extremely randomized trees, the regression algorithm that produces the paper's best predictions.","marker":"Geurts et al. (2006)"},{"why":"Introduces the Boruta feature-selection algorithm used to remove irrelevant WISE variables before training.","marker":"Kursa & Rudnicki (2010)"},{"why":"Provides the power transformation applied to normalize the WISE predictor variables.","marker":"Yeo & Johnson (2000)"},{"why":"Documents disagreements between WISE-based and Spitzer-based infrared-excess studies that the predicted fluxes are meant to resolve.","marker":"Patel et al. (2014)"}],"fun_headline_variants":["ML turns WISE data into Spitzer-quality fluxes","Extreme trees predict Spitzer-grade IR fluxes from WISE","AI fills gap between WISE and Spitzer mid-IR measurements","ML maps WISE to Spitzer photometry with R² up to 0.98","Spitzer-quality fluxes from WISE alone via machine learning"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method's generalization rests on the assumption that the WISE-to-Spitzer relationship learned from clean, high-quality open cluster members in the SEIP catalog holds for the broader allWISE population, including fainter, redder, or more embedded sources with SEIP counterparts.","fun_headline_variants_meta":{"raw":{"variants":["ML turns WISE data into Spitzer-quality fluxes","Extreme trees predict Spitzer-grade IR fluxes from WISE","AI fills gap between WISE and Spitzer mid-IR measurements","ML maps WISE to Spitzer photometry with R² up to 0.98","Spitzer-quality fluxes from WISE alone via machine learning"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000244,"raw_usage":{"total_tokens":1618,"prompt_tokens":1115,"completion_tokens":503,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":731,"completion_tokens_details":{"reasoning_tokens":414}},"tokens_in":731,"tokens_out":503,"duration_ms":5757,"temperature":1.0,"reasoning_tokens":414,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T16:32:34.459024+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a set of allWISE sources that do have SEIP Spitzer detections but were not used in training because they are not open-cluster members or fail the clean-sample flags, apply the trained model, and compare predicted against measured 8 µm and 24 µm fluxes; if $R^2$ falls well below the reported 0.98 and 0.94 values, or if a systematic bias appears at fluxes below about $10^3$ µJy, the generalization claim fails.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the census of 3530 open clusters with Gaia DR3 membership probabilities that defines the HR24 training sample."},{"cited_title":"W., Roellig, T","cited_arxiv_id":null,"evidence_quote":"Describes the Spitzer mission and its IRAC and MIPS instruments, defining the target fluxes the model tries to predict."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Introduces the Boruta feature-selection algorithm used to remove irrelevant WISE variables before training."},{"cited_title":"& Johnson, R","cited_arxiv_id":null,"evidence_quote":"Provides the power transformation applied to normalize the WISE predictor variables."},{"cited_title":"I., Metchev, S","cited_arxiv_id":null,"evidence_quote":"Documents disagreements between WISE-based and Spitzer-based infrared-excess studies that the predicted fluxes are meant to resolve."}],"review_version":1}