{"id":"35f3ef12-84e6-43e2-8ef2-072fb5c040c7","arxiv_id":"2411.08747","paper_version":2,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"A normalizing-flow-based regression method and a KS-test variant estimate intrinsic distributions and error correlations from data; mock tests show they beat standard methods in low-S/N log-linear regression.","lead":"This paper presents a regression method that uses normalizing flows to learn the unobservable intrinsic distribution of the independent variable and its correlation with uncertainty levels directly from noisy data. It reports that the method, especially a KS-test based variant, outperforms common astronomical regression tools in mock tests with low signal-to-noise ratios and complex errors.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The NF-estimated intrinsic distribution and error correlation are plugged into the likelihood as known; the paper never validates that these latent components are recovered, so the claimed low-S/N advantage could rest on a misspecified likelihood.","rationale":"The reader's weakest assumption is exactly the same: the NF-derived P(x) and P(x_err|x) are treated as known. This is load-bearing because both proposed methods (ML and KS) use these estimates in the likelihood or in the generative model for the KS objective. If the estimates are biased, the improvement over LINMIX/Leopy at low S/N could be an artifact of the mock setup rather than a genuine gain. The paper provides only an observable-level goodness-of-fit (2D KS), which cannot distinguish between different latent decompositions. The oracle test I propose would directly separate the NF's contribution from the regression framework's contribution, settling whether the assumption holds in the regime where the central claim is made. The reader's conditional verdict should stand, with this test as a natural condition for acceptance. I do not see a reason to reject or strengthen the verdict based on this concern alone; it reinforces the need for the conditional requirement rather than overturning the paper's demonstrated mock performance.","tokens_in":26101,"tokens_out":7460,"duration_ms":70141,"concrete_test":"Run the ML method on 20 Sample-1 datasets (N=3000) using the paper's NF pipeline; then rerun it with the true P(x) and P(x_err|x) used in the mock generator (oracle), keeping all other settings identical. If the median absolute error in k, b, σ changes by less than, say, 0.01, the NF assumption is safe; if it changes by more than the reported improvement over LINMIX, the central claim is not established. Also report the same comparison for the KS-test method.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim depends on the empirical Bayes step (Eq. 8–9) recovering the true P(x) and P(x_err|x). The paper's only NF validation is a 2D KS test on the observable (x_obs, x_err); this does not certify the latent decomposition, which is identifiable only under assumptions not stated. With finite N and a flexible spline NF, the recovered P(x), P(x_err|x) may differ from the truth while still matching observables. The likelihood Eq. 3 then uses a misspecified P(x) and P(x_err|x), biasing θ, especially at low S/N where these terms matter most. Figure 12's degradation below N=300 is consistent with this, but no test at the claimed N>1000 regime isolates the NF contribution. The paper also reports manual tuning of NF hyperparameters in >5% of datasets, introducing researcher degrees of freedom.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes two regression methods for astronomical data with measurement errors in both variables, an unknown intrinsic distribution of the independent variable, and correlations between uncertainty levels and intrinsic values. The ML-based method uses a normalizing flow to estimate these nuisance distributions in an empirical-Bayes step, then maximizes the resulting likelihood. The KS-test based variant replaces the likelihood with a 2D Kolmogorov-Smirnov p-value as the optimization objective. Both are tested on extensive mock data covering nonlinearity, complex intrinsic distributions, outliers, saturation, and inaccurate uncertainties, and are applied to PHANGS-ALMA/JWST CO-MIR flux correlations. The central claim is that for sample sizes above 1000, especially at low signal-to-noise ratios, the new methods outperform several established methods (OLS, WLS, ODR, wODR, mODR, LINMIX, LtsFit, Leopy).","tokens_in":26280,"tokens_out":4931,"duration_ms":41802,"significance":"If validated, the methods address a genuine gap: existing Bayesian regression tools such as LINMIX, ROXY, and t-cup rely on Gaussian mixtures and ignore uncertainty-level correlations, while Leopy requires the intrinsic distribution as user input. The mock evaluation is extensive, uses known ground truth (so the performance evidence is external, not circular), and covers a wide range of realistic failure modes. The paper also includes coverage tests and a clear sample-size-dependent recommendation. The code will be made public upon acceptance. However, the central claim rests on the empirical-Bayes step recovering the latent P(x) and P(x_err|x), and this latent recovery is not directly validated; this is the main risk to the soundness of the proposed methods.","major_comments":[{"comment":"The likelihood in Eq. (3) treats the NF-estimated P(x; phi_2) and P(x_err|x; phi_1) as known. The only validation reported is a 2D KS test comparing the joint observable distribution (x_obs, x_err) to that generated by the fitted NF. This does not certify that the latent decomposition into P(x) and P(x_err|x) is correct; different latent pairs can lead to the same observable joint distribution, especially when the error distribution is flexible. Since the likelihood and the KS generative model use these latent densities directly, a misspecified NF can bias theta in a way that the observable-only KS test may not detect. I ask the authors to add a direct validation of the recovered latent densities on mocks with known truth, or a sensitivity test in which the NF is intentionally misspecified, to demonstrate that the low-S/N advantage is not an artifact of the empirical-Bayes plug-in.","section":"Section 2.1, Eqs. (7)-(9)"},{"comment":"The paper states that in more than 95% of datasets the default NF configuration yields a 2D KS p-value above 3e-3, and that 'for the cases where the 2D KS test fails, we manually tune the NF hyperparameters to pass the test.' This manual tuning introduces researcher degrees of freedom into the reported results; for up to 5% of the datasets the final model is not produced by the default algorithm. Since the headline claims include extreme-error fractions of 0 (Figure 2), the fraction of manually tuned datasets should be reported per sample, and the analysis should be repeated without those datasets to show that the conclusions do not depend on manual intervention.","section":"Section 3.1"},{"comment":"The KS-test based method optimizes the p-value of a 2D KS test, which the authors themselves note 'lacks a rigorous mathematical foundation.' The grid-search adaptation and the use of p-values as posterior weights are heuristic. Given that the KS method is recommended as first choice for N>1000, the paper should provide evidence that this objective is well-behaved: for example, a small simulation showing the p-value surface is unimodal, or that the grid search converges to the same solution from different initial ranges. Without this, it is unclear whether the reported robustness of the KS method is a property of the method or of the particular grid-search implementation.","section":"Section 2.1"}],"minor_comments":[{"comment":"The definition of the extreme error fraction eta (residuals exceeding 1.0 for k and b, and 0.5 for sigma) first appears only in the Figure 1 caption; it would be helpful to define it in the main text.","section":"Section 3.2.1"},{"comment":"There are several typographical issues, e.g., 'T able 1' in Section 2.3, 'with espect to' in Section 4.2, and subject-verb agreement errors ('exhibits' vs. 'exhibit' in Sections 3.2.7 and 3.3).","section":"Throughout"},{"comment":"The statement that 'different randomly selected subsamples yield highly similar results' is not quantified; please provide the scatter of the fitted parameters across subsamples.","section":"Section 4.2"},{"comment":"The coverage test is only performed on three methods; since the paper emphasizes the KS method's robustness, a coverage test on the other compared methods might be informative, but this is optional.","section":"Section 3.2.9"}],"recommendation":"major_revision","confidential_remarks":"The paper is a strong methods contribution with extensive mock testing, but the two load-bearing issues - direct validation of the latent NF decomposition and the manual tuning in more than 5% of datasets - need to be addressed before publication. The code is not yet available for inspection; I would encourage the editor to request it as part of the revision. Note also that the 2D KS objective is acknowledged by the authors to lack a rigorous foundation, so the KS variant's performance claims would be strengthened by a calibration study."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know. First, this is a real methods paper, not a repackaging: it replaces GMMs or user-supplied P(x) with a normalizing flow estimated from the data, and it models P(x_err|x), which most tools ignore. The mock tests are the strongest part. Second, the central low-S/N claim rests on an empirical-Bayes step that is never directly validated, and the code isn't public yet. Both are fixable.\n\nThe likelihood derivation is standard but competently extended. The test suite is broad: 13 intrinsic distributions, several noise models, outliers, saturation, sample-size scaling. Results are consistent, and the KS variant's robustness is impressive. The coverage analysis is a nice touch. They also cite prior work explicitly and correctly.\n\nThe latent NF issue is the one that matters. Equations 8-9 estimate P(x) and P(x_err|x) from (x_obs, x_err); the only check is a 2D KS on observables. That does not certify the latent decomposition. In mocks the method does well across many P(x), which is reassuring, but the paper doesn't show recovered versus true P(x) or isolate the NF contribution. Since the headline advantage is at low S/N where these nuisance terms matter most, a referee should ask for that test. Minor: the 2D KS objective is heuristic, the posterior approximation for KS is approximate (they say so), and manual tuning of NF hyperparameters in more than 5% of datasets is a reproducibility concern. Also, a methods paper should not leave the code to acceptance; a public release with a reproducibility script would clear up a lot.\n\nBottom line: this deserves a serious referee. The contribution is solid and the weaknesses are fixable. I'd ask for code release, a latent-recovery validation on mocks, and a clearer statement of when the KS method's grid search is reliable.","headline":"A genuine advance in empirical-Bayes regression for astronomical data, with an unusually thorough mock-test campaign; the latent-distribution validation and code release need attention before I'd trust the headline low-S/N claim.","tokens_in":26757,"tokens_out":1898,"would_cite":true,"duration_ms":19734,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that a normalizing-flow-based maximum-likelihood regression, and a 2D KS-test variant, recover unbiased slopes, intercepts, and intrinsic scatter from astronomical data with unknown intrinsic distributions and error…","keywords":["regression","normalizing flows","measurement errors","empirical Bayes","intrinsic scatter","log-linear relation","Kolmogorov-Smirnov test","astronomical catalogs"],"falsifier":"Take a mock sample with $N=2000$, a known bimodal $P(\\log x)$, $k=0.5$, $b=0$, $\\sigma=0.3$, typical S/N of 1 for both variables, and known Gaussian errors; run the public raddest implementation with the NF given too few training epochs so that its 2D-KS goodness-of-fit p-value is below $10^{-3}$. If the recovered $(k,b,\\sigma)$ are then as biased as LINMIX's, the claim that the method itself, rather than a well-fitted NF, delivers the advantage would be called into question.","tokens_in":25916,"feed_emoji":"🔭","tokens_out":8275,"duration_ms":63641,"temperature":0.7,"pith_summary":"The paper sets out to solve a persistent problem in astronomy: fitting a scaling relation when both variables carry measurement errors, the true distribution of the independent variable is unknown, and the size of the error depends on the value being measured. The authors propose estimating those unknown pieces from the observed data with a normalizing flow, then maximizing the resulting likelihood (or, in a variant, the p-value of a two-dimensional KS test between generated and observed data). On mock log-linear datasets with sample sizes above 1000, they report that both variants beat OLS, WLS, ODR, wODR, mODR, LINMIX, LtsFit, and Leopy, with the largest gains at signal-to-noise ratios near unity. The KS-test variant is reported to stay accurate under non-log-linear trends, complex intrinsic distributions, outliers, and saturation effects, which are common in real data. A sympathetic reader would care because the method removes the need to guess the intrinsic distribution or to assume a linear-space Gaussian error model.","feed_headline":"Normalizing-flow regression beats standard fits on noisy data","feed_subtitle":"Learns the unknown intrinsic distribution and error correlations from the data; wins at low S/N and on PHANGS maps.","key_machinery":"The load-bearing object is the normalizing flow: a bijective map $f_\\phi$ that turns the variable of interest into a latent variable $z$ with a tractable density, so the modeled density is $P_\\phi(x)=P_z(f_\\phi(x))\\,|\\det J_{f_\\phi}(x)|$. The paper uses two such flows, one for the pair $(x_{\\rm err},x)$ and one for $(y_{\\rm err},y)$, each factorized as $P(x_{\\rm err}|x;\\phi_1)P(x;\\phi_2)$, and fits them by maximizing the marginal likelihood of the observed $(x_{\\rm obs},x_{\\rm err})$ pairs, with the latent $x$ integrated out by Gauss-Hermite quadrature — a variational-inference/empirical-Bayes step. The fitted densities enter the regression likelihood as if they were known. In the KS-test variant, the same generative machinery produces synthetic $(x_{\\rm obs},y_{\\rm obs})$ data under trial parameters, and the optimization objective is the p-value of the 2D Kolmogorov-Smirnov test comparing synthetic and observed data; because that p-value is not differentiable, the search is done by iterative grid refinement. This is the mechanism that lets the method avoid assuming a Gaussian mixture for $P(x)$ and avoid requiring the user to supply the intrinsic distribution.","core_discovery":"The central claim is that the bias that plagues regression with noisy independent variables can be removed by learning the nuisance distributions from the data itself rather than assuming them. Specifically, the paper argues that a normalizing flow trained on the observed pairs $(x_{\\rm obs}, x_{\\rm err})$ can recover the latent intrinsic distribution $P(x)$ and the conditional error-level distribution $P(x_{\\rm err}|x)$ through the empirical-Bayes marginal likelihood in Eq. (9); plugging these estimates into the full likelihood of Eq. (3) then yields maximum-likelihood estimates of the slope $k$, intercept $b$, and intrinsic scatter $\\sigma$ that are unbiased for log-linear relations, provided the sample is large enough (roughly $N>1000$). The KS-test variant replaces the likelihood objective with the p-value of a 2D KS test between data generated by the model and the observed data, and is claimed to be markedly more robust when the model assumptions are violated. The paper further claims that for $300\\lesssim N\\lesssim 1000$ the ML variant remains the best performer, while for $N<300$ it is competitive with, though no better than, LINMIX and Leopy.","pith_inferences":["Editorial inference: the two-step empirical-Bayes design means the quality of the regression is gated by the quality of the normalizing-flow fit; the paper's own sample-size results are consistent with this, so a practical prescription would be to require a goodness-of-fit check on the flow before trusting the regression output.","Editorial inference: because the likelihood in Eq. (3) is written for a multidimensional independent variable and normalizing flows scale to high dimensions, the same machinery should extend to bivariate or multivariate scaling relations, where Gaussian-mixture-based methods struggle.","Editorial inference: the KS objective, being a distribution-distance criterion, could be reused outside regression — for example, to fit parameters of physical models to noisy observations in other domains where likelihoods are misspecified.","Editorial inference: when $N<300$, the method's advantage disappears; a user with a small sample might do just as well with a simpler method, and a hybrid that injects prior knowledge of $P(x)$ into the flow could extend the regime."],"forward_implications":["At sample sizes above about 1000, the method is claimed to outperform all eight comparison methods on mock log-linear data, with the KS variant recording an extreme-error fraction of zero for slope, intercept, and scatter across 200 mock datasets.","The KS variant is claimed to be robust to violations of the log-linear assumption, complex intrinsic distributions, complex uncertainty-intrinsic correlations, misestimated uncertainties, outliers, and saturation, so it can be applied where standard methods degrade.","For samples of roughly 300–1000, the ML variant is claimed to retain the best median absolute errors, making it the recommended choice in that regime.","The posterior uncertainties from the new methods are closer to calibrated than LINMIX's (coverage-area 0.09 and 0.10 vs 0.15), though all three remain imperfect.","On PHANGS-ALMA and PHANGS-JWST maps of CO(2-1) versus mid-infrared flux, the KS method gives steeper slopes and smaller intercepts than mODR and LINMIX, differences the paper traces to saturation at high flux, noise in the independent variable, and the log-linear versus linear error model."],"supporting_citations":[{"why":"Supplies the LINMIX baseline and the likelihood form for regression with errors in both variables that this method generalizes.","marker":"Kelly 2007"},{"why":"Supplies the Leopy baseline and the alternative approach that requires the intrinsic distribution as user input, which the NF removes.","marker":"Feldmann 2019"},{"why":"Supplies the ROXY baseline and the Gaussian-mixture treatment of P(x) that the NF replaces.","marker":"Bartlett & Desmond 2023"},{"why":"Supplies the t-cup baseline, robustness via Student's t, and the extreme-deconvolution alternative for P(x).","marker":"Martin & Mortlock 2024"},{"why":"Foundational normalizing-flow construction that the density estimation is built on.","marker":"Dinh et al. 2014"},{"why":"Provides the normalizing-flow density formula used to model P(x) and P(x_err|x).","marker":"Jimenez Rezende & Mohamed 2015"},{"why":"Defines the 2D Kolmogorov-Smirnov test used both for NF goodness-of-fit and as the KS regression objective.","marker":"Peacock 1983"},{"why":"Supplies the mODR method and the PHANGS-based real-data application that the paper re-analyzes.","marker":"Leroy et al. 2023"},{"why":"Provides the PZFlow package that the authors customize to implement the normalizing flow.","marker":"Crenshaw et al. 2024a,b"}],"fun_headline_variants":["Flow-based regression learns error correlations, beats standard fits","KS-test regression robust to bad assumptions, beats LINMIX","Learning intrinsic scatter from data beats assuming it","Unbiased regression with learned distributions","Raddest: flow-based fitting beats LINMIX on noisy data"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the normalizing flow trained on observed $(x_{\\rm obs},x_{\\rm err})$ pairs gives an accurate estimate of the latent intrinsic distribution $P(x)$ and the conditional error distribution $P(x_{\\rm err}|x)$, and that these estimates can be treated as known when the regression likelihood is evaluated; if the flow is undertrained or too rigid, the likelihood is misspecified and the regression parameters can be biased, which is consistent with the paper's finding that the advantage shrinks below about 300 samples.","fun_headline_variants_meta":{"raw":{"variants":["Flow-based regression learns error correlations, beats standard fits","KS-test regression robust to bad assumptions, beats LINMIX","Learning intrinsic scatter from data beats assuming it","Unbiased regression with learned distributions","Raddest: flow-based fitting beats LINMIX on noisy data"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000967,"raw_usage":{"total_tokens":4172,"prompt_tokens":1058,"completion_tokens":3114,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":674,"completion_tokens_details":{"reasoning_tokens":3038}},"tokens_in":674,"tokens_out":3114,"duration_ms":21952,"temperature":1.0,"reasoning_tokens":3038,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T21:23:22.480838+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a mock sample with $N=2000$, a known bimodal $P(\\log x)$, $k=0.5$, $b=0$, $\\sigma=0.3$, typical S/N of 1 for both variables, and known Gaussian errors; run the public raddest implementation with the NF given too few training epochs so that its 2D-KS goodness-of-fit p-value is below $10^{-3}$. If the recovered $(k,b,\\sigma)$ are then as biased as LINMIX's, the claim that the method itself, rather than a well-fitted NF, delivers the advantage would be called into question.","supporting_citations":[],"review_version":1}