REVIEW 2 major objections 6 minor 67 references
A hybrid network that reads galaxy colors as a wavelength sequence cuts photometric-redshift errors by about 10% and outliers by about 20% on an LSST proxy sample.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · grok-4.5
2026-07-10 14:41 UTC pith:D4SC4IVY
load-bearing objection Clean, controlled ~10–20% gains over the Jones BNN on the public GalaxiesML test set with released code; useful incremental photo-z tool whose main limit is the acknowledged spectroscopic selection function. the 2 major comments →
Enhancing Photometric Redshift Estimation for LSST with a Hybrid LSTM-Mixture Density Network
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
On the identical GalaxiesML test set used by the prior Bayesian neural-network benchmark, the LSTM-MDNz architecture improves every standard point-estimate metric by roughly 10% and reduces both ordinary and catastrophic outlier fractions by roughly 20%, while producing well-calibrated multimodal posterior PDFs whose probability-integral-transform distribution is statistically consistent with uniform.
What carries the argument
LSTM-MDNz: multi-band magnitudes and errors are arranged as a wavelength-ordered sequence of length five, passed through two bidirectional LSTM layers that extract SED-like gradients, then mapped by a mixture-density head (K=10 Gaussians) onto a full posterior PDF.
Load-bearing premise
The spectroscopically selected, bright-end-weighted GalaxiesML catalog is assumed to be a fair enough proxy that the measured gains will still hold for the much fainter, higher-redshift galaxies that will dominate LSST cosmology samples.
What would settle it
Retrain and re-evaluate the identical architecture on a deeper, purely photometric-selected sample whose magnitude and color distributions match the expected LSST weak-lensing source population; if the relative gains versus the same BNN baseline vanish or reverse, the claim fails.
If this is right
- Catalogs filtered at modest z_conf thresholds can meet or exceed LSST outlier-rate requirements while retaining more than 90% of the sample.
- Well-calibrated multimodal PDFs can be stacked or marginalized directly into n(z) estimates without the extra dispersion corrections needed for over-broad unimodal Gaussians.
- The same sequential-plus-MDN recipe is immediately portable to any multi-band survey whose filters can be ordered by wavelength.
- Because photometric uncertainties are ingested as explicit sequence features, the model automatically down-weights low-S/N bands without hand-tuned inverse-variance weights.
Where Pith is reading between the lines
- The same architecture should transfer with little retuning to Euclid or CSST once their filter sequences replace the HSC g,r,i,z,y ladder.
- Domain-adversarial training or SOM re-weighting of the spectroscopic training set would be the natural next step to close the bright-to-faint gap the authors themselves flag.
- If z_conf is treated as a continuous weight rather than a hard cut, it could enter likelihoods for cosmic shear or BAO analyses without discarding objects.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces and validates LSTM-MDNz, a hybrid architecture that treats multi-band HSC photometry (and magnitude errors) as wavelength-ordered sequences processed by stacked Bi-LSTMs, then models the redshift posterior as a 10-component Gaussian mixture via an MDN. On the identical GalaxiesML public test set used by Jones et al. (2024), the model improves point-estimate metrics (RMSE 0.145 o0.130, MAE 0.055 o0.048, σ_NMAD 0.026 o0.024) and reduces outlier and catastrophic-outlier rates by ~18–22% relative to the BNN baseline, while producing a near-uniform PIT (KS=0.014) and lower CRPS. A PDF-based confidence metric z_conf is shown to enable high-purity catalog construction by discarding a few percent of low-confidence objects. An ablation (magnitudes only) and redshift-binned diagnostics are provided.
Significance. If the reported gains hold under the stated experimental controls, the work supplies a practical, lightweight (~3.8×10^5 parameters), open-source probabilistic photo-z pipeline that improves both accuracy and calibration over a recent BNN baseline on a widely used LSST-proxy catalog. The public code/weights, fixed Zenodo test set, magnitude-only ablation, multi-metric tables, PIT/CRPS diagnostics, and z_conf purity curves are concrete strengths that make the empirical claims reproducible and useful for the community. The domain-shift caveat (spectroscopic selection vs. faint LSST depths) is acknowledged by the authors and correctly limits the cosmological claim, but does not erase the value of a carefully controlled architectural comparison on a public benchmark.
major comments (2)
- §2.1 and the final discussion correctly flag that GalaxiesML is bright-end weighted, incomplete at i≳23.5 and in the 1.2<z<2 desert, and therefore only a partial proxy for the faint, high-z LSST cosmological sample. The central claim of “enhancing photo-z for LSST” therefore rests on an untested transfer assumption. The manuscript should either (a) add a quantitative domain-shift test (e.g., magnitude- or color-matched faint subsample, or comparison against a deeper photometric reference such as COSMOS2020 TransferZ as in Soriano et al.), or (b) systematically soften the LSST-readiness language in the abstract, title framing, and conclusions so that the claim is scoped strictly to the controlled GalaxiesML benchmark.
- Table 1 and §4.2: the magnitude-only ablation already shows that sequential architecture alone is competitive with the BNN; the full model’s further gains come from feeding cmodel magnitude errors. Because the BNN baseline of Jones et al. is described as using primarily cmodel magnitudes (without the same explicit error channels), part of the reported ~10–20% improvement may be attributable to richer inputs rather than architecture. A controlled re-run of the BNN (or an MLP-MDN) with identical magnitude+error inputs would isolate the architectural contribution and should be added or explicitly discussed as a remaining ambiguity.
minor comments (6)
- Abstract and §1: “~10% improvement … across RMSE, MAE, scatter, and σ_NMAD” slightly overstates the scatter/σ_NMAD gains (0.026 o0.023/0.024); the precise percentages already given for RMSE/MAE/outliers are preferable.
- §3.1 / Eq. (2): the choice K=10 is stated without a sensitivity study; a short note or appendix showing that results are stable for K∈{5,10,15} would strengthen the free-parameter claim.
- §4.4: α=0.05 for the z_conf window is said to be robust over [0.03,0.15], but the supporting numbers are not shown; a one-sentence table or parenthetical would help.
- Figure 2 caption and text: “Relative Point Density” color scale is clear, but the exact kernel/binning used for the density map is not stated.
- Typographical: “server as a small-scale proxy” (abstract/intro), “magenitude error”, occasional missing spaces around z_conf thresholds, and “Ph t metric” / “Spectr sc pic” artifacts in figure labels should be cleaned.
- References: the prior quasar LSTM-MDNz paper (Chen et al. 2026) is appropriately cited as the methodological source; ensure the GalaxiesML and Jones et al. Zenodo DOIs remain prominent for reproducibility.
Circularity Check
No circularity: empirical ML performance claims are measured against external spectroscopic ground truth on a held-out public test set; architecture self-citations supply only the method, not the galaxy results.
full rationale
The paper's load-bearing claims are purely empirical: on the identical GalaxiesML test set (DOI 10.5281/zenodo.10145347) released by Jones et al. (2024), LSTM-MDNz yields lower RMSE/MAE/σ_NMAD, lower outlier fractions, lower CRPS, and a near-uniform PIT (KS=0.014) relative to the BNN baseline, plus an ablation (Mags Only) and z_conf purity curves. All metrics are computed against independent spectroscopic redshifts after training on a disjoint split; none of the reported numbers is forced by construction from a fitted parameter or definition. The architecture (Bi-LSTM + MDN with K=10) and the z_conf definition (integral of the PDF over a scaled window around the mean) are taken from the authors' prior quasar paper (Chen et al. 2026) and standard PDF-morphology practice, but those citations supply only the reusable method; the galaxy results, the comparison to Jones et al., and the quantitative gains are new measurements on an external benchmark. No uniqueness theorem, ansatz that embeds the target metric, or self-definitional loop appears. Domain-shift caveats are acknowledged by the authors themselves and do not constitute circularity. The derivation chain is therefore self-contained against external data.
Axiom & Free-Parameter Ledger
free parameters (4)
- number of Gaussian mixture components K =
10
- z_conf window half-width alpha =
0.05
- Bi-LSTM hidden sizes and dropout =
128-64 / 0.25
- learning-rate schedule and early-stopping patience
axioms (4)
- domain assumption Multi-band cmodel magnitudes ordered by wavelength form a sequence whose sequential correlations encode SED physics useful for redshift.
- ad hoc to paper A ten-component GMM is flexible enough to capture the relevant multi-modal and heavy-tailed photo-z posteriors.
- domain assumption The GalaxiesML spectroscopic sample, despite known selection biases, is an adequate small-scale proxy for LSST-like photo-z performance.
- standard math Negative log-likelihood of the GMM is the appropriate training objective for calibrated PDFs.
invented entities (1)
-
z_conf confidence metric
independent evidence
read the original abstract
Accurate photometric redshift (photo-$z$) estimation and robust uncertainty quantification are essential for the LSST to achieve its precision cosmology goals. Traditional machine learning algorithms are largely restricted to point estimates, struggling to characterize the multimodal nature of redshift PDFs and the degeneracies within the color-redshift space. To address this, we present and validate the LSTM-MDNz architecture, which integrates sequential feature extraction with flexible probability density modeling to enhance both prediction accuracy and uncertainty calibration across a broad redshift range, thereby meeting the stringent data quality requirements necessitated by next-generation cosmological analysis. The LSTM-MDNz framework treats multi-band photometry as wavelength-ordered sequences, utilizing LSTM networks to capture non-linear evolutionary correlations across the SED. A Mixture Density Network (MDN) is then employed to explicitly model posterior PDFs via Gaussian mixture models (GMMs). Performance is evaluated on the HSC GalaxiesML dataset (which serves as a small-scale proxy for next-generation surveys like LSST) and benchmarked against the BNN architecture established by Jones et al. (2024). The proposed model consistently outperforms the BNN baseline, achieving a $\sim 10\%$ improvement in point-estimation accuracy (specifically across RMSE, MAE, scatter, and $\sigma_{\text{NMAD}}$) and a $\sim 20\%$ reduction in the rates of both general and catastrophic outliers. A uniform probability integral transform (PIT) distribution confirms well-calibrated probabilistic outputs. Furthermore, the PDF-based confidence metric $z_{\text{conf}}$ enables high-purity catalog construction: excluding just approximately $4\%$ of extremely low-confidence ($z_{\text{conf}} < 0.05$) samples reduces the overall outlier rate by $\sim 48\%$.
Figures
Reference graph
Works this paper leans on
-
[1]
The Wide Field Infrared Survey Telescope: 100 Hubbles for the 2020s
Akeson, R., Armus, L., Bachelet, E., et al. 2019, arXiv e-pri nts, arXiv:1902.05569
work page internal anchor Pith review Pith/arXiv arXiv 2019
-
[2]
Alam, S., Albareti, F. D., Allende Prieto, C., et al. 2015, Ap JS, 219, 12
work page 2015
- [3]
-
[4]
Arnouts, S., Cristiani, S., Moscardini, L., et al. 1999, MNR AS, 310, 540
work page 1999
-
[5]
Beck, R., Lin, C.-A., Ishida, E. E. O., et al. 2017, MNRAS, 468 , 4323
work page 2017
-
[6]
Practical recommendations for gradient-based training of deep architectures
Bengio, Y . 2012, arXiv e-prints, arXiv:1206.5533 Benítez, N. 2000, ApJ, 536, 571
work page internal anchor Pith review Pith/arXiv arXiv 2012
- [7]
- [8]
- [9]
-
[10]
Bradshaw, E. J., Almaini, O., Hartley, W. G., et al. 2013, MNR AS, 433, 194
work page 2013
-
[11]
Brammer, G. B., van Dokkum, P . G., & Coppi, P . 2008, ApJ, 686, 1503
work page 2008
- [12]
-
[13]
2018, MNRAS, 480, 2178 Carrasco Kind, M
Cao, Y ., Gong, Y ., Meng, X.-M., et al. 2018, MNRAS, 480, 2178 Carrasco Kind, M. & Brunner, R. J. 2013, MNRAS, 432, 1483
work page 2018
- [14]
- [15]
- [16]
-
[17]
Cool, R. J., Moustakas, J., Blanton, M. R., et al. 2013, ApJ, 7 67, 118
work page 2013
- [18]
-
[19]
Cooper, M. C., Gri ffith, R. L., Newman, J. A., et al. 2012, MNRAS, 419, 3018 CSST Collaboration, Gong, Y ., Miao, H., et al. 2026, Science China Physics, Mechanics, and Astronomy, 69, 239501 Article number, page 16 Zhijian Luo et al.: Enhancing Photometric Redshift Estimat ion for LSST with a Hybrid LSTM-Mixture Density Network
work page 2012
- [20]
-
[21]
Davis, M., Faber, S. M., Newman, J., et al. 2003, in Society of Photo-Optical Instrumentation Engineers (SPIE) Conference Series, V ol. 4834, Discover- ies and Research Prospects from 6- to 10-Meter-Class Telesc opes II, ed. P . Guhathakurta, 161–172
work page 2003
-
[22]
Dawid, A. P . 1984, Journal of the Royal Statistical Society: Series A (General), 147, 278 D’Isanto, A. & Polsterer, K. L. 2018, A&A, 609, A111
work page 1984
-
[23]
Do, T., Boscoe, B., Jones, E., Li, Y . Q., & Alfaro, K. 2024, arX iv e-prints, arXiv:2410.00271
work page internal anchor Pith review Pith/arXiv arXiv 2024
-
[24]
Drinkwater, M. J., Jurek, R. J., Blake, C., et al. 2010, MNRAS , 401, 1429 Euclid Collaboration, Desprez, G., Paltani, S., et al. 2020 , A&A, 644, A31 Euclid Collaboration, Mellier, Y ., Abdurro’uf, et al. 2025 , A&A, 697, A1
work page 2010
-
[25]
Feldmann, R., Carollo, C. M., Porciani, C., et al. 2006, MNRA S, 372, 565
work page 2006
- [26]
-
[27]
Garilli, B., Guzzo, L., Scodeggio, M., et al. 2014, A&A, 562, A23
work page 2014
-
[28]
Graham, M. L., Connolly, A. J., Ivezi ´c, Ž., et al. 2018, AJ, 155, 1
work page 2018
- [29]
- [30]
-
[31]
Hsieh, B. C. & Y ee, H. K. C. 2014, ApJ, 792, 102
work page 2014
-
[32]
2006, MNRA S, 366, 101 Ivezi´c, Ž., Kahn, S
Huterer, D., Takada, M., Bernstein, G., & Jain, B. 2006, MNRA S, 366, 101 Ivezi´c, Ž., Kahn, S. M., Tyson, J. A., et al. 2019, ApJ, 873, 111
work page 2006
- [33]
- [34]
- [35]
-
[36]
Khostovan, A. A., Kartaltepe, J. S., Salvato, M., et al. 2026 , ApJS, 282, 6
work page 2026
-
[37]
Kingma, D. P . & Ba, J. 2014, arXiv e-prints, arXiv:1412.6980
work page internal anchor Pith review Pith/arXiv arXiv 2014
-
[38]
Kitching, T. D., Miller, L., Heymans, C. E., van Waerbeke, L. , & Heavens, A. F. 2008, MNRAS, 390, 149
work page 2008
-
[39]
Euclid Definition Study Report
Laureijs, R., Amiaux, J., Arduini, S., et al. 2011, arXiv e-prints, arXiv:1110.3193 Le Fèvre, O., Cassata, P ., Cucciati, O., et al. 2013, A&A, 559 , A14
work page internal anchor Pith review Pith/arXiv arXiv 2011
- [40]
-
[41]
J., Le Brun, V ., Maier, C., et al
Lilly, S. J., Le Brun, V ., Maier, C., et al. 2009, ApJS, 184, 21 8
work page 2009
-
[42]
LSST Science Book, Version 2.0
Liske, J., Baldry, I. K., Driver, S. P ., et al. 2015, MNRAS, 45 2, 2087 LSST Science Collaboration, Abell, P . A., Allison, J., et al. 2009, arXiv e-prints, arXiv:0912.0201
work page internal anchor Pith review Pith/arXiv arXiv 2015
- [43]
- [44]
- [45]
-
[46]
McLure, R. J., Pearce, H. J., Dunlop, J. S., et al. 2013, MNRAS , 428, 1088
work page 2013
-
[47]
Momcheva, I. G., Brammer, G. B., van Dokkum, P . G., et al. 2016 , ApJS, 225, 27
work page 2016
-
[48]
Mucesh, S., Hartley, W. G., Palmese, A., et al. 2021, MNRAS, 5 02, 2770
work page 2021
- [49]
-
[50]
Newman, J. A. & Gruen, D. 2022, ARA&A, 60, 363
work page 2022
-
[51]
Photometric Redshifts for the Hyper Suprime-Cam Subaru Strategic Program Data Release 2
Nishizawa, A. J., Hsieh, B.-C., Tanaka, M., & Takata, T. 2020 , arXiv e-prints, arXiv:2003.01511 Pâris, I., Petitjean, P ., Aubourg, É., et al. 2018, A&A, 613, A51
work page internal anchor Pith review Pith/arXiv arXiv 2020
-
[52]
Pasquet, J., Bertin, E., Treyer, M., Arnouts, S., & Fouchez, D. 2019, A&A, 621, A26
work page 2019
-
[53]
Uncertain Photometric Redshifts
Polsterer, K. L., D’Isanto, A., & Gieseke, F. 2016, arXiv e-p rints, arXiv:1608.08016
work page internal anchor Pith review Pith/arXiv arXiv 2016
-
[54]
M., Seitz, S., Brimioulle, F., et al
Rau, M. M., Seitz, S., Brimioulle, F., et al. 2015, MNRAS, 452 , 3710
work page 2015
- [55]
-
[56]
Schmidt, S. J. & Thorman, P . 2013, MNRAS, 431, 2766
work page 2013
-
[57]
D., Kashino, D., Sanders, D., et al
Silverman, J. D., Kashino, D., Sanders, D., et al. 2015, ApJS , 220, 12
work page 2015
- [58]
-
[59]
Skelton, R. E., Whitaker, K. E., Momcheva, I. G., et al. 2014, ApJS, 214, 24
work page 2014
- [60]
-
[61]
Soriano, J., Saikrishnan, S., Seenivasan, V ., et al. 2024, a rXiv e-prints, arXiv:2411.18054
work page internal anchor Pith review Pith/arXiv arXiv 2024
- [62]
- [63]
-
[64]
Wyatt, M. & Singal, J. 2021, PASP , 133, 044504 Y oo, J., Gyure, C., Agarwal, V ., Singal, J., & Silverman, G. 2026, ApJ, 998, 258
work page 2021
-
[65]
2011, Scientia Sinica Physica, Mechanica & Astrono mica, 41, 1441
Zhan, H. 2011, Scientia Sinica Physica, Mechanica & Astrono mica, 41, 1441
work page 2011
-
[66]
2022, Research in Astr onomy and As- trophysics, 22, 115017
Zhou, X., Gong, Y ., Meng, X.-M., et al. 2022, Research in Astr onomy and As- trophysics, 22, 115017
work page 2022
-
[67]
2021, ApJ, 909, 53 Article number, page 17
Zhou, X., Gong, Y ., Meng, X.-M., et al. 2021, ApJ, 909, 53 Article number, page 17
work page 2021
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.