{"id":"0e4c3271-522f-4d18-a493-b3e42ddd1da2","arxiv_id":"2501.06532","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":8,"one_line_summary":"CGANs can estimate photometric redshifts and redshift probability distributions from four galaxy magnitudes, reaching accuracies close to Mixture Density Networks on Dark Energy Survey data.","lead":"A researcher tested whether a type of artificial intelligence called a conditional GAN can estimate galaxy distances from brightness measurements alone. The method gives results nearly as good as the current best technique, though not quite better, and could offer a new way to handle uncertainty in these estimates.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The CGAN-vs-MDN comparison for PDFs depends on an unstated procedure for converting generator samples into per-galaxy densities; without it, the central comparability claim is not reproducible.","rationale":"I read the paper as a proof-of-concept: CGANs can produce photo-z point estimates and PDFs whose aggregate quality is close to an MDN baseline on DES-Y1. For that claim to hold, the reported CGAN PDFs must be well-defined outputs of the trained model. They are not, as written: the mapping from generator samples (Eq. 5) to a continuous density phi(z) used in Eqs. 8-10 is unspecified, and the same is true for the point estimate used in Section 3.1. This is not a disagreement with the community consensus; it is an internal reproducibility gap in the evaluation. The reader's weakest assumption identified exactly this PDF-estimator issue, and I agree. I do not see a reason to move the verdict: the point-estimate comparison is plausible, the code is promised on GitHub, and the missing details are discoverable or fixable in revision. A conditional acceptance that requires the density-estimation procedure and point-estimation statistic to be specified (or shown to not affect results) is proportionate. I also note that the paper itself states MDN is slightly superior and CGAN PDFs are overconfident in the coverage test, so the 'comparable' claim is already modest; the concern is about how that claim is supported, not about fraud or fundamental unsoundness.","tokens_in":8308,"tokens_out":4331,"duration_ms":44672,"concrete_test":"Inspect the public GitHub repository (github.com/mgarciafernandez-uem/CGAN-photoz) and identify the exact procedure used to turn generator outputs into densities and point estimates: number of z draws per galaxy, density estimator, bandwidth/bins, and aggregation statistic. Then rerun the Section 3.2 metrics on the DES-Y1 validation split with the identified procedure and with at least two variants (e.g., N=100 vs N=1000 draws; KDE bandwidth 0.01 vs 0.05). If the PIT, Odds, or Coverage curves move by more than the bootstrap scatter shown in Figures 4-5, the published comparison is estimator-dominated and the 'comparable quality metrics' claim needs to be qualified; if they are insensitive, the omission is a reproducibility issue only.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is that CGAN photometric redshift PDFs are comparable to MDN PDFs (Abstract, Section 4). Section 3.2 evaluates this via PIT, Odds, and Coverage (Eqs. 8-10), all of which require a per-galaxy conditional density phi(z) for each CGAN prediction. In a CGAN, phi(z) is only available implicitly through repeated draws yhat_i(z_j) = G(z_j | x_i) (Section 2.1, Eq. 5). The manuscript never states how many latent draws are used per galaxy, what density estimator is applied (histogram, KDE, kernel shape, bandwidth/bins), or what support the density is evaluated on. The same gap affects the point estimates in Section 3.1: G is stochastic, so the reported yhat_i must be a statistic (mean, median, or single draw) of G(z | x_i), and this is not specified. Consequently, Figures 3-6 and all quoted metrics may reflect the arbitrary choice of the density estimator rather than the trained CGAN distribution. The GitHub repository may contain the missing implementation, but as submitted the evaluation described in the text is incomplete. This is the load-bearing weak point because the proof-of-concept conclusion explicitly extends to probability density estimation, not just point estimates.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript presents a proof-of-concept application of Conditional Generative Adversarial Networks (CGANs) to photometric redshift estimation. The CGAN is trained with an f-divergence objective (KL divergence) on DES-Y1 Stripe-82 galaxies with griz MAG_AUTO magnitudes, using an 80/10/10 train/test/validation split of 33,410 galaxies. The generator produces stochastic redshift draws conditioned on magnitudes, and these draws are used both for point estimation and, implicitly, for per-galaxy probability density functions. The model is compared with a Mixture Density Network (MDN) based on Ansari et al. (2021). Point-estimate metrics (mean absolute bias, sigma_NMAD, outlier fraction) and PDF metrics (PIT, Odds, coverage, stacked distributions) are evaluated on a held-out validation sample, with bootstrap confidence intervals. The reported result is that the MDN is slightly better but the CGAN metrics are comparable, establishing CGANs as a viable non-parametric alternative for photo-z estimation.","tokens_in":8582,"tokens_out":5794,"duration_ms":53481,"significance":"The significance of this work is as a proof of concept: if the comparison is credible, CGANs offer a way to estimate photometric-redshift PDFs without assuming a parametric family, which is potentially useful for current and future wide-field surveys. The paper's strengths include the use of a held-out validation set, bootstrap confidence intervals, a fixed external benchmark MDN, and a publicly available code repository. The central weakness is that the construction of PDFs from generator samples is not described, so the PDF-comparability claim is not currently reproducible from the text. The work is not circular: training and validation are separated, and no parameter-free derivation or model-independent prediction is claimed.","major_comments":[{"comment":"The PDF metrics PIT, Odds, and Coverage all require a per-galaxy conditional density phi(z). For the CGAN this density is only available implicitly through repeated draws yhat_i(z_j) = G(z_j | x_i) (Sec. 2.1, Eq. 5). The manuscript never states how many latent draws are used per galaxy, what density estimator is applied (histogram, KDE, kernel shape, bandwidth, binning), or on what support and integration grid phi(z) is evaluated. Because Eqs. (8)-(10) and the stacked PDF in Fig. 6 depend on this unspecified estimator, all quoted calibration results could reflect the estimator choice rather than the trained CGAN distribution. This is the load-bearing gap for the paper's PDF-comparability claim; the companion repository may contain the details, but the submitted text is incomplete.","section":"Sec. 3.2, Eqs. (8)-(10)"},{"comment":"G(z | x_i) is stochastic, yet the point estimates shown in Fig. 3 and used in Eqs. (6)-(7) require a definite value of yhat_i for each galaxy. The paper does not state whether the reported point estimate is a single Monte Carlo draw, the mean, or the median of G(z | x_i). Since the point-estimate metrics and their bootstrap intervals in Fig. 4 depend on this choice, the comparison with the MDN point estimates is not fully specified.","section":"Sec. 3.1, Eq. (5)"},{"comment":"The text states that quality metrics are computed over the validation sample, but the bootstrap description says 'generating 1,000 bootstrap samples from the test dataset.' Earlier, the test set is described as being used only to monitor overfitting during training. If the bootstrap resamples the test set rather than the validation set, the quoted confidence intervals are not computed on the same sample as the point estimates. Please clarify which sample is used and, if necessary, recompute the intervals on the validation sample.","section":"Sec. 3.1, bootstrap paragraph"},{"comment":"No numerical values are reported for any quality metric; the support for the claim of 'comparable quality' is entirely visual. Please provide a table with the point-estimate and PDF metrics (for example, mean |Delta z|, sigma_NMAD, outlier fraction, PIT statistics, odds statistics, and coverage deviations) together with bootstrap intervals. Without numbers, a reader cannot judge whether the observed differences between CGAN and MDN are statistically significant or practically important.","section":"Sec. 3, Figs. 4-5"}],"minor_comments":[{"comment":"The abstract and introduction describe this as the 'first' algorithmic approach to photometric redshifts using CGANs, but no literature search or comparison with prior CGAN-based photo-z work is provided. Please either document the novelty claim or soften the wording.","section":"Abstract and Introduction"},{"comment":"The nested notation in Eq. (2), such as D(G(z_i | x_i); theta_G) | x_i; theta_D, is confusing; rewriting the discriminator as D(y, x; theta_D) and D(G(z | x), x; theta_D) would improve readability.","section":"Sec. 2.1, Eq. (2)"},{"comment":"The statement that Wasserstein GANs 'do not allow the interpretation of outputs as probability densities' is too categorical and is not established by the cited reference; please clarify what property is meant and how it follows from the Wasserstein formalism.","section":"Sec. 2.1"},{"comment":"The matching details with SDSS Stripe-82 spectroscopic redshifts are missing, including the matching radius, any redshift quality flags, and the treatment of duplicate or repeated observations. Please specify these choices.","section":"Sec. 3"},{"comment":"The caption says the CGAN loss functions were shifted by +1 to avoid negative values, but the axes of the loss panels are not labeled. Please state explicitly which quantity is plotted on each axis.","section":"Fig. 2 caption"},{"comment":"No random seed or reproducibility setting is reported for the data split, initialization, or training. Since GAN training is stochastic, a random seed or a statement about the number of independent training runs would help reproducibility.","section":"Sec. 3"}],"recommendation":"major_revision","confidential_remarks":"I did not inspect the companion GitHub repository; the missing PDF-estimation details may be present there, in which case the authors should move them into the main text. The 'first' claim and the test/validation bootstrap inconsistency should be checked carefully before acceptance. The manuscript is within the scope of astro-ph.IM and the central idea is worth publishing after a major revision that specifies the density-estimation procedure and reports numerical metrics."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear colleague,\n\nThis paper is a reasonable first demonstration that a CGAN can produce photometric redshift point estimates and PDFs competitive with an MDN on DES-Y1 data. The novel bit is real: nobody else in the cited literature has applied CGANs to photo-z, and the f-divergence training framework is a sensible way to get conditional densities without a parametric form. The comparison to MDN is honest and the conclusion is measured—MDN is slightly better on most metrics—which I take as a good sign.\n\nWhat the paper does well: the point-estimate metrics (bias, NMAD, outlier rate) are computed on a held-out validation set with bootstrap confidence intervals, and the stacked PDF comparison in Figure 6 is a nice sanity check. The authors give their code and the MDN baseline they used, so the point-estimate part is largely reproducible.\n\nThe soft spot is exactly what the stress-test flags: the conversion from generator samples to per-galaxy PDFs is left unspecified. The CGAN output is a stochastic function G(z|x); to get a point estimate you need a statistic (mean, median, or a single draw), and no such statistic is stated. To get a PDF for the PIT, Odds, and coverage metrics, you need a density estimator (histogram, KDE, kernel width, binning) over multiple noise draws, and the paper says none of that. Since the PDF claim is half of the proof-of-concept, this is a load-bearing omission. It is fixable—the GitHub code likely contains the details—but as submitted the evaluation in Section 3.2 is incomplete. Also, the text waves between \"test\" and \"validation\" in the bootstrap description; minor.\n\nI don't think this kills the paper. The point-estimate comparison stands, and the PDF comparison is likely salvageable once the procedure is specified. The central claim—that a CGAN can be trained for photo-z without assuming a functional form and yields comparable results—holds up as a proof of concept, though the sample size and single survey limit generalization.\n\nWho should read this: anyone working on generative models for photo-z, or people building ML pipelines for DES/LSST who want to know what the GAN baseline looks like. It is not a breakthrough method, but it is a legitimate first step.\n\nRecommendation: send it to peer review, but ask the authors to specify the sample-to-density procedure and the point estimate statistic before it is accepted. The reviewer should also verify that the bootstrap is done on the validation set.\n\nBest,\n[Your name]","headline":"A plausible first proof-of-concept for CGAN photo-z, with an honest MDN comparison, but the PDF evaluation is underspecified to the point of being non-reproducible as written.","tokens_in":9098,"tokens_out":2565,"would_cite":false,"duration_ms":24104,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper proposes the first use of CGANs for photometric redshift estimation: on DES-Y1 data their point estimates and PDFs are close to a Mixture Density Network, though the MDN remains slightly more accurate.","keywords":["photometric redshifts","Conditional Generative Adversarial Networks","Mixture Density Networks","Dark Energy Survey","probability density estimation","f-divergence","galaxy surveys","redshift calibration"],"falsifier":"Re-run the same DES-Y1 experiment with a fully documented PDF estimator, for example 1,000 generator draws per galaxy and a Gaussian kernel density estimate with a stated bandwidth, and recompute the PIT, Odds, and coverage curves; if the coverage curve still falls below the expected $1-\\alpha$ by more than the bootstrap uncertainty band, the claim of PDF quality comparable to the MDN would fail under a fixed, reproducible estimator.","tokens_in":8094,"feed_emoji":"🌌","tokens_out":8651,"duration_ms":65610,"temperature":0.7,"pith_summary":"This paper establishes that Conditional Generative Adversarial Networks (CGANs), trained through the f-divergence formalism, can serve as a non-parametric method for estimating galaxy photometric redshifts, producing both point estimates and full probability density functions. On Dark Energy Survey Y1 data matched to spectroscopic redshifts, the CGAN's point-estimation metrics (bias, normalized median absolute deviation, outlier rate) and density-calibration metrics (PIT, Odds, coverage) come close to those of a Mixture Density Network, although the MDN remains slightly more accurate and less outlier-prone. The significance is practical: photometric redshifts are a key input for wide-field cosmological surveys, and most current density-estimating methods assume a parametric form such as Gaussians or Gaussian mixtures, whereas a CGAN learns the conditional density directly from data. The paper presents the approach as a proof of concept that adversarial networks are viable for this task.","feed_headline":"Adversarial networks rival the leading galaxy-redshift method","feed_subtitle":"Non-parametric CGANs deliver photo-z point estimates and PDFs close to Mixture Density Networks on DES-Y1 data.","key_machinery":"The load-bearing object is a Conditional Generative Adversarial Network with generator $G(z|x)$ and discriminator $D(y|x)$, trained by the f-divergence variational objective with the KL divergence, where the activation pair is $g_f(x)=x$ and $f^*(g_f(x))=e^{x-1}$. The generator draws a random vector $z$ and, conditioned on the four MAG_AUTO magnitudes in griz bands, outputs a photometric redshift $\\hat{y}$; the discriminator classifies an input redshift as real spectroscopic or generated. This adversarial game makes $\\hat{y}$ a random sample from the conditional density $\\phi(z|x)$, so the method produces point estimates and PDFs without assuming a functional shape, in contrast with the Mixture Density Network baseline, which assumes a mixture of 30 Gaussians. The PDF quality metrics (PIT, Odds, Coverage) are then computed from these generated samples.","core_discovery":"On its own terms, the paper claims that a CGAN whose generator maps a random vector and four griz magnitudes to a photometric-redshift sample, and whose discriminator tries to separate real spectroscopic redshifts from generated ones, learns the conditional distribution of redshift given photometry. Using the KL-divergence within the f-GAN variational framework, the generator outputs are draws from the conditional density, so the same network yields both a point estimate and a per-galaxy PDF without specifying a parametric family. Tested on 33,410 DES-Y1 galaxies with spectroscopic redshifts, the CGAN traces the true redshift distribution and achieves quality metrics close to the MDN baseline across all redshift bins, with the MDN showing slightly higher accuracy and the CGAN producing more outliers and slightly overconfident PDFs. The paper concludes that this is a proof of concept that CGANs are a viable alternative for photometric redshift estimation and opens the door to further exploration of adversarial networks in this field.","pith_inferences":["The unstated number of generator draws and the unstated density estimator are a hidden degree of freedom: the reported PIT and coverage curves could shift by an amount comparable to the CGAN-MDN gap depending on how the empirical density is built, so a reproducibility appendix specifying them would materially strengthen the comparison.","Because each galaxy's output is a stochastic sample, the CGAN naturally encodes redshift uncertainty and could be embedded in downstream weak-lensing or clustering analyses that need to propagate photo-z errors without assuming Gaussian shapes.","The architecture should transfer to upcoming surveys with more photometric bands by simply enlarging the input layer, and the non-parametric density may be most valuable in parts of color space where galaxy types mix and multi-modal redshift distributions occur.","A direct testable extension would be to train the same CGAN with a different f-divergence, for example Jensen-Shannon, and check whether the observed overconfidence is a property of the KL choice or of the adversarial training itself."],"forward_implications":["A single non-parametric CGAN can produce both point estimates and full PDFs for photometric redshifts, removing the need to specify a Gaussian or mixture shape in advance.","On DES-Y1 data, the CGAN's point-estimation metrics and PIT/Odds distributions are close to the MDN's, so adversarial training is a viable alternative despite the MDN's small advantage in accuracy and outlier rate.","The coverage test shows the CGAN PDFs are overconfident, identifying calibration as the main remaining weakness of the method.","The stacked CGAN probability densities give a similar representation of the underlying redshift distribution as the MDN, though the MDN matches the spectroscopic data more closely.","Conditioning on galaxy type and using larger samples, as the paper proposes for future work, should reduce the heterogeneity that limits current precision."],"supporting_citations":[{"why":"introduces the generative adversarial network framework that the CGAN extends.","marker":"[11]"},{"why":"defines conditional GANs, giving the conditioning-on-magnitudes mechanism.","marker":"[24]"},{"why":"provides the f-divergence variational training objective and the KL activation pair used in the loss functions.","marker":"[27]"},{"why":"supplies the Mixture Density Network implementation and configuration used as the comparison baseline.","marker":"[1]"},{"why":"defines the PIT, Odds, and Coverage metrics used to evaluate the probability density estimates.","marker":"[34]"},{"why":"justifies excluding Wasserstein GANs on the grounds that their outputs are not interpretable as probability densities.","marker":"[33]"},{"why":"supplies the coverage-test tools for conditional density estimation used in the calibration comparison.","marker":"[7]"},{"why":"defines the Odds metric for probability density estimation quality.","marker":"[20]"}],"fun_headline_variants":["CGANs nearly match MDN for galaxy photo-z","Adversarial networks close in on top photo-z method","Generative model offers competitive galaxy photo-z","Conditional GANs rival standard photo-z accuracy","Photo-z from CGAN: close to the leading approach"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The evaluation of the probability-density outputs assumes that drawing a finite number of random noise vectors from the trained generator and turning them into a density with some estimator faithfully represents the CGAN's conditional distribution; the paper does not state the number of draws per galaxy or the density-estimation procedure, so the PDF-calibration metrics rest on that unstated choice.","fun_headline_variants_meta":{"raw":{"variants":["CGANs nearly match MDN for galaxy photo-z","Adversarial networks close in on top photo-z method","Generative model offers competitive galaxy photo-z","Conditional GANs rival standard photo-z accuracy","Photo-z from CGAN: close to the leading approach"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001276,"raw_usage":{"total_tokens":5194,"prompt_tokens":896,"completion_tokens":4298,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":512,"completion_tokens_details":{"reasoning_tokens":4221}},"tokens_in":512,"tokens_out":4298,"duration_ms":87670,"temperature":1.0,"reasoning_tokens":4221,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T20:57:40.174472+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the same DES-Y1 experiment with a fully documented PDF estimator, for example 1,000 generator draws per galaxy and a Gaussian kernel density estimate with a stated bandwidth, and recompute the PIT, Odds, and coverage curves; if the coverage curve still falls below the expected $1-\\alpha$ by more than the bootstrap uncertainty band, the claim of PDF quality comparable to the MDN would fail under a fixed, reproducible estimator.","supporting_citations":[{"cited_title":"In: Proceedings of the 27th International Conference on Neural Information Processing Systems - Volume","cited_arxiv_id":null,"evidence_quote":"introduces the generative adversarial network framework that the CGAN extends."},{"cited_title":"Annual Review of Astronomy and Astrophysics60, 363–414 (Aug 2022)","cited_arxiv_id":null,"evidence_quote":"provides the f-divergence variational training objective and the KL activation pair used in the loss functions."},{"cited_title":"A&A 650, A90 (2021)","cited_arxiv_id":null,"evidence_quote":"supplies the Mixture Density Network implementation and configuration used as the comparison baseline."},{"cited_title":"Astronomy and Computing49, 100886 (2024)","cited_arxiv_id":null,"evidence_quote":"defines the PIT, Odds, and Coverage metrics used to evaluate the probability density estimates."},{"cited_title":"Nature Astronomy 3, 212–222 (Jun 2019)","cited_arxiv_id":null,"evidence_quote":"justifies excluding Wasserstein GANs on the grounds that their outputs are not interpretable as probability densities."},{"cited_title":"Astronomy and Computing30, 100362 (2020)","cited_arxiv_id":null,"evidence_quote":"supplies the coverage-test tools for conditional density estimation used in the calibration comparison."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"defines the Odds metric for probability density estimation quality."}],"review_version":1}