{"id":"cbbdb5ad-5cff-4576-a557-577cc7e55a41","arxiv_id":"2509.01478","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A cross-validated family of moment estimators, indexed by kappa and spanning NLS, Poisson, and gamma PML, often beats Poisson regression on sparse, heteroskedastic non-negative finance data.","lead":"This paper shows that defaulting to Poisson regression for count-like financial data can be wasteful when the data has both many zeros and variance that grows with the average. It introduces an estimator family spanning gamma, Poisson, and least squares, and uses cross-validation to choose the best member, which often beat Poisson on four finance datasets.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Empirical RMSE gains are inflated by selecting κ on the same test set used to measure performance.","rationale":"The reader's weakest_assumption was the asymmetric censoring model and its role in the bias ordering. That is a valid theoretical concern, but the paper's practical contribution is the CV selector, and the empirical magnitudes are the most striking part of the abstract and conclusion. The test-set selection issue is a more immediate, concrete threat to the central claim: it can be checked directly and, if confirmed, would materially reduce the credibility of the reported 50–90% improvements. The theoretical framework may still be correct under the assumed DGP, and simulations can support the trade-off, but the real-data evidence for the CV procedure's value needs to be based on honest out-of-sample evaluation. The reader already noted this in their rationale, so there is partial agreement; however, they did not make it the weakest assumption. If the nested CV test shows large gains persist, the concern disappears and the verdict could be strengthened. If the gains shrink, the paper would need to moderate its applied claims and present the applications as illustrations rather than definitive evidence. Hence CONDITIONAL is appropriate, contingent on a corrected evaluation.","tokens_in":32602,"tokens_out":5034,"duration_ms":60099,"concrete_test":"Re-run the four empirical analyses with nested cross-validation. For each of B=200 outer 80/20 random splits, select κ via inner 5-fold CV on the training 80% using the same grid as in Figure 9, then evaluate the chosen estimator on the untouched test 20%. Compute the average outer-test RMSE for the selected κ and for κ=0, the percentage reduction, and a bootstrap standard error across outer splits. If the reduction is substantially below 50–90% or not statistically significant, the headline empirical claim is an artifact of test-set selection.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central applied claim that cross-validation selects a generalized PML estimator with 50–90% lower out-of-sample RMSE than Poisson (Section 5.2) is supported by an evaluation protocol that is circular. The authors average test-fold RMSEs over 2000 random 80/20 splits, then select κ as the minimizer of this averaged test RMSE and report the difference between this minimum and the Poisson (κ=0) point on the same curve. Because the RMSE curve is estimated with noise, the minimum is a biased (optimistic) estimate of the achievable RMSE for the selected model. The same test data are used for both model selection and performance evaluation, so the reported improvements are effectively in-sample for the selection rule. This does not invalidate the theoretical bias-variance analysis, but it means the practical benefit of the proposed CV procedure for real finance datasets remains unquantified. The paper itself lacks error bars on these improvements, and two of the four outcomes are truncated, further obscuring the true effect. A proper nested evaluation is required before the 50–90% figures can be taken at face value.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper argues that Poisson PML, although widely recommended for non-negative/count outcomes, can be suboptimal when both heteroskedasticity and sparsity are present. It proposes a one-parameter family of generalized PML estimators defined by the moment equations (6), which nests Poisson (κ=0), NLS (κ=1), and gamma PML (κ=-1). The paper derives approximate bias and asymptotic variance formulas under an asymmetric censoring model (3)-(4) and heteroskedasticity (2), showing a bias-variance trade-off indexed by κ. It recommends selecting κ by cross-validation and reports, in four finance datasets, that the selected estimator can reduce out-of-sample RMSE by 50-90% relative to Poisson PML.","tokens_in":32774,"tokens_out":10541,"duration_ms":117272,"significance":"If the results hold, the paper provides a useful framework and a practical data-driven alternative to the default use of Poisson PML. Its strengths are the explicit estimating-equation family (6), the asymptotic bias/variance analysis in Theorems 1-2, and extensive simulations (5000 replications) that convincingly demonstrate a trade-off under the assumed censoring model. The cross-validation proposal is simple and actionable. However, the headline empirical claims are currently overstated because the evaluation protocol is circular and the data pre-processing is not fully justified; these issues are fixable but require substantive revisions.","major_comments":[{"comment":"The reported 50-90% out-of-sample RMSE improvements are computed by evaluating the test-fold RMSE curve over 2000 random 80/20 splits, selecting the κ that minimizes that same curve, and comparing the minimum with the Poisson point (κ=0) on the same curve. The minimum of a noisy estimated curve is an optimistically biased estimator of the selected model's achievable RMSE; this is effectively in-sample selection. No error bars are given. Please re-evaluate with nested cross-validation (select κ on training folds, evaluate on a separate test split) and report standard errors or confidence intervals for the improvement.","section":"§5.2"},{"comment":"The patents and permits outcomes are apparently 'truncated to samples with y≤25' and 'y≤50'. If this truncation is applied to the estimation/evaluation sample, the target is a censored outcome and the RMSE comparisons (and the optimal κ) are not for the original economic variable. The extreme 90% gain for gamma PML on permits is especially sensitive to this choice. Please clarify whether truncation is only for plotting; if it is used in the analysis, justify it or use the full sample with a loss appropriate for the actual outcome.","section":"§5.1 / Figure 8"},{"comment":"The stated denominator for the NLS bias, ∫p(X)X²(1+P)exp(2θ0ᵀX), conflicts with Theorem 1, Eq. (9), which for κ=1 gives A = ∫(P−1)xxᵀp exp(2θ0ᵀX) = −∫(1−P)xxᵀp exp(2θ0ᵀX). The sign of the censoring adjustment matters for the conclusion that NLS is more robust than PPML. In addition, Lemma 1 in Appendix F.2 appears to use scalar denominators E[exp(2θ0ᵀX)] and drops the xxᵀ factor; please reconcile with Theorem 2.","section":"Appendix C, Corollary 1"},{"comment":"'As heteroskedasticity decreases or sparsity increases, the optimal κ decreases' contradicts the paper's own argument (Section 3.1, Figure 7) that greater sparsity favors larger κ. If read literally, this gives the opposite practical advice. Please correct and verify the direction in Figure 7.","section":"§4, final paragraph"},{"comment":"The bias-reduction property of κ>0 is derived under the asymmetric censoring model (3)-(4), where P(Y=0|X) decreases monotonically in exp(θ0ᵀX). The paper's own Appendix E.1 notes that the bias approximation is 'less accurate for the gamma PML.' The general claim that Poisson 'can be far from optimal' is only established for this class of DGPs. Please add a robustness simulation with structural zeros (e.g., zero-inflation with P(Y=0|X) independent of or increasing in the mean) to show where the recommendation reverses, and soften the general wording accordingly.","section":"§2.2 / §3.2"}],"minor_comments":[{"comment":"'procedureselects' is missing a space.","section":"Abstract"},{"comment":"'To To contextualize' should be 'To contextualize'.","section":"§2.1"},{"comment":"The default data description says January 2019-December 2023, but Figure 8 caption says January 2020-December 2023. Please reconcile.","section":"§5.1"},{"comment":"The plots label κ=-0.95 as 'gamma PML'; this should be stated clearly in the text, not only in parentheses.","section":"Appendix E.1"},{"comment":"The censoring probability 1/(1+exp(βθ0ᵀx)) used in the appendix differs from the main specification 1/(1+(τ exp(θ0ᵀx))^β) in (4). Please clarify the relationship.","section":"Appendix E.1 / Figure E.10"}],"recommendation":"major_revision","confidential_remarks":"The paper's theoretical framework and simulation evidence are promising, but the headline 50-90% empirical improvement is likely inflated by the circular evaluation protocol and possibly by outcome truncation. The mismatch between Theorem 1 and Corollary 1 also needs correction before the theoretical comparison can be trusted. I recommend major revision rather than reject because the central idea is sound and the empirical claims can be redone with proper nested CV and full-sample analysis."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The framework is worth your time; the headline numbers are not. The paper gives a systematic bias-variance analysis of the kappa-weighted PML family under asymmetric zero-censoring, and that part is a genuine contribution. The family itself is a reparametrization of the classical power-variance PML family, so the estimators are not new. What is new is the explicit first-order bias approximation (Theorem 1), the censoring-adjusted variance formula (Theorem 2), and the phase-transition characterization of when different kappa values dominate. The simulations corroborate the direction of the theory, and the four finance applications show that the selected kappa can vary a lot across datasets and can change coefficient conclusions. That is a useful message for applied researchers, and the paper honestly credits Santos Silva and Tenreyro and the existing PML literature.\n\nThe soft spots are real. Most important: the 50-90% out-of-sample RMSE improvements are computed by selecting kappa as the minimizer of the averaged test-fold RMSE over the same 2000 splits, then reporting the gap between that minimum and the Poisson point on the same curve. The test data are doing double duty, so those improvements are in-sample for the selection rule. A nested CV or a genuinely separate holdout would be needed to quantify the real benefit. The paper also truncates two of the four outcome variables (patents at y<=25, permits at y<=50); the RMSE gains and the coefficient comparisons in Table 2 are therefore conditional on that truncation and may not generalize to the full distribution. The authors acknowledge in Appendix E.1 that the bias approximation is less accurate for gamma PML, which is a limitation for the theoretical coverage. And the qualitative bias ordering is largely baked into the weighting: increasing kappa mechanically downweights censored small-mean observations, so the direction of the trade-off is not surprising. The value is in the quantification and the data-driven selector, not in the discovery that Poisson is not always best.\n\nThe stress-test note about circular evaluation holds up on reading. The rest of the framework is coherent and the paper is clearly written. It deserves a serious referee, but the referee should push for an honest nested evaluation and for handling the truncation before the empirical magnitude claims are accepted.","headline":"The kappa-family bias-variance framework is a real contribution, but the headline 50-90% RMSE improvements are optimistic because kappa is selected on the same test folds used to measure them, and two outcomes are truncated.","tokens_in":33351,"tokens_out":2909,"would_cite":true,"duration_ms":32355,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["62F12","62J12","62P20"],"pacs":[],"model":"deepseek-v4-flash","headline":"Poisson regression can be far from optimal for sparse, heteroskedastic count-like data; a one-parameter family of moment estimators with cross-validated kappa cuts out-of-sample RMSE by 50–90% across four finance datasets.","keywords":["count-like data","heteroskedasticity","sparsity","pseudo maximum likelihood","Poisson regression","bias-variance trade-off","cross-validation","non-negative outcomes"],"falsifier":"Simulate data with Var(Y|X)=E^alpha(Y|X) but with a zero probability P(Y=0|X) that is constant or increasing in the conditional mean (for example, structural zeros from a separate binary process). If the cross-validated kappa then selects values near 0 or below, and Poisson has lower out-of-sample MSE than kappa=1, the paper's monotone-censoring-driven bias ordering is falsified.","tokens_in":32331,"feed_emoji":"📊","tokens_out":5400,"duration_ms":55223,"temperature":0.7,"pith_summary":"The paper takes aim at the near-default use of Poisson pseudo-maximum-likelihood (PPML) regression for non-negative outcomes in finance and economics. It argues that when outcomes are both sparse (many zeros) and heteroskedastic, PPML sits at a poor fixed point of a bias-variance trade-off: estimators that weight large-mean observations reduce censoring bias, while estimators that weight small-mean observations gain efficiency. The authors build a one-parameter family of moment estimators that spans Poisson, gamma PML, and non-linear least squares, and show that the best member depends on the relative strength of sparsity and heteroskedasticity. A simple cross-validation selection procedure picks the member; in simulations and four finance datasets it beats Poisson, often by large margins and with different economic conclusions.","feed_headline":"Poisson isn't optimal for sparse count-like finance data","feed_subtitle":"A one-parameter estimator family cuts out-of-sample RMSE by 50–90% versus Poisson on four datasets.","key_machinery":"The generalized pseudo-maximum-likelihood (PML) estimating equations: (1/n) sum (y_i - exp(theta^T x_i)) exp(kappa theta^T x_i) x_i = 0, for kappa in [-1,1] (or all real kappa). They nest Poisson (kappa=0), NLS (kappa=1), and gamma PML (kappa=-1), and reduce to negative binomial when a c=1 shift is included. The exponent kappa tunes the weight given to observations with larger conditional means, and the population version of the equations, under an asymmetric censoring model where P(Y=0|X) decreases with exp(theta_0^T X), yields the bias and variance formulas that drive the trade-off; k-fold cross-validation on out-of-sample MSE then selects kappa.","core_discovery":"The central claim is that the optimal estimator in the family summing (y_i - exp(theta^T x_i)) exp(kappa theta^T x_i) x_i = 0 is often not the Poisson member (kappa=0) when zeros arise from asymmetric censoring of an underlying exponential-mean process. Bias falls as kappa rises because the exp(kappa theta^T x) weight downweights small-mean observations that are disproportionately censored; variance rises with kappa under heteroskedasticity. Theorems 1 and 2 give asymptotic bias and variance approximations for the family, showing bias independent of the heteroskedasticity index alpha and variance depending on alpha, kappa, and censoring. Cross-validation over kappa selects the MSE-minimizing","pith_inferences":["If excess zeros are generated by structural zeros (e.g., firms that never innovate) rather than censoring of small positive values, the monotone-zero-probability assumption fails and the recommended kappa>0 ordering may reverse; one can check whether P(Y=0|X) declines with the fitted mean before adopting the CV choice.","Because the uncensored variance-minimizing kappa is 1-alpha under Var(Y|X)=E^alpha(Y|X), a practitioner who estimates alpha can restrict the CV grid or interpret the selected kappa as an implied trade-off point between censoring bias and heteroskedasticity.","The same estimating-equation family with a general f(theta^T x) in place of exp(theta^T x) (Appendix D) could extend the bias-variance logic beyond log-linear conditional means to other nonlinear specifications."],"forward_implications":["In data with many zeros and mild heteroskedasticity, NLS (kappa=1) or other kappa>0 estimators can beat Poisson, reversing the common advice to avoid NLS for non-negative outcomes.","The same cross-validation procedure can be applied to trade, health, or other non-negative datasets; the selected kappa, not a default Poisson, should define the benchmark model.","Parameter estimates and standard economic conclusions can change materially with kappa (e.g., ratings coefficient -0.646 to -2.565, patents overconfidence 0.617 to 0.125), so model choice affects inference, not only prediction.","The phase-transition map (optimal kappa as a function of alpha and tau) gives applied researchers a diagnostic: stronger sparsity pushes optimal kappa up, stronger heteroskedasticity pushes it down."],"supporting_citations":[{"why":"Establishes that PML estimators are consistent for the conditional mean and gives the efficiency benchmark that the paper re-derives as kappa = 1 - alpha.","marker":"Gourieroux et al. (1984b)"},{"why":"Baseline evidence that Poisson PML is robust to heteroskedasticity and rounding zeros; the paper extends this to simultaneous heteroskedasticity and sparsity.","marker":"Santos Silva and Tenreyro (2006)"},{"why":"Simulation evidence that Poisson outperforms gamma PML under zeros; the paper generalizes the comparison to the full kappa family.","marker":"Santos Silva and Tenreyro (2011)"},{"why":"Shows economic conclusions can flip between Poisson and log-linear; supplies the replicated corporate-patents dataset used in Section 5.","marker":"Cohn et al. (2022)"},{"why":"Original corporate-patents application whose dataset is replicated for the patents outcome.","marker":"Hirshleifer et al. (2012)"},{"why":"Provides the predictor set for the corporate-default application.","marker":"Duffie et al. (2007)"},{"why":"Source of the residential-permits dataset used in Section 5.","marker":"Bekkerman et al. (2023)"},{"why":"Zero-inflated Poisson model; one of the asymmetric censoring models the paper compares against.","marker":"Lambert (1992)"}],"fun_headline_variants":["Sparse finance data? Rethink Poisson regression","One-parameter family slashes RMSE over Poisson on sparse data","Beyond Poisson: flexible estimator for sparse finance data","Sparse data in finance: Poisson may cost you 90% RMSE","Cross-validated estimator beats Poisson on sparse finance data"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The ordering of estimators relies on zeros being censored versions of small underlying means, so that the chance of a zero falls as the conditional mean rises; if many zeros are structural and do not decrease with the mean, the recommended kappa>0 can be wrong.","fun_headline_variants_meta":{"raw":{"variants":["Sparse finance data? Rethink Poisson regression","One-parameter family slashes RMSE over Poisson on sparse data","Beyond Poisson: flexible estimator for sparse finance data","Sparse data in finance: Poisson may cost you 90% RMSE","Cross-validated estimator beats Poisson on sparse finance data"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00085,"raw_usage":{"total_tokens":3478,"prompt_tokens":636,"completion_tokens":2842,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":380,"completion_tokens_details":{"reasoning_tokens":2758}},"tokens_in":380,"tokens_out":2842,"duration_ms":21643,"temperature":1.0,"reasoning_tokens":2758,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T12:32:13.840053+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Simulate data with Var(Y|X)=E^alpha(Y|X) but with a zero probability P(Y=0|X) that is constant or increasing in the conditional mean (for example, structural zeros from a separate binary process). If the cross-validated kappa then selects values near 0 or below, and Poisson has lower out-of-sample MSE than kappa=1, the paper's monotone-censoring-driven bias ordering is falsified.","supporting_citations":[{"cited_title":", author Tenreyro, S","cited_arxiv_id":null,"evidence_quote":"Baseline evidence that Poisson PML is robust to heteroskedasticity and rounding zeros; the paper extends this to simultaneous heteroskedasticity and sparsity."},{"cited_title":", author Tenreyro, S","cited_arxiv_id":null,"evidence_quote":"Simulation evidence that Poisson outperforms gamma PML under zeros; the paper generalizes the comparison to the full kappa family."},{"cited_title":", author Liu, Z","cited_arxiv_id":null,"evidence_quote":"Shows economic conclusions can flip between Poisson and log-linear; supplies the replicated corporate-patents dataset used in Section 5."},{"cited_title":", author Low, A","cited_arxiv_id":null,"evidence_quote":"Original corporate-patents application whose dataset is replicated for the patents outcome."},{"cited_title":", author Saita, L","cited_arxiv_id":null,"evidence_quote":"Provides the predictor set for the corporate-default application."},{"cited_title":", author Cohen, M.C","cited_arxiv_id":null,"evidence_quote":"Source of the residential-permits dataset used in Section 5."},{"cited_title":", year 1992","cited_arxiv_id":null,"evidence_quote":"Zero-inflated Poisson model; one of the asymmetric censoring models the paper compares against."}],"review_version":1}