{"id":"89b8d8aa-216b-499f-baf8-ec102dc7a4e8","arxiv_id":"2611.14719","paper_version":1,"verdict":"ACCEPT","confidence":"HIGH","novelty_score":7.0,"correctness_risk":"low","formal_verification":"none","parameter_count":5,"one_line_summary":"Recursive least squares updates define a martingale posterior for regression, yielding an order-invariant posterior mean and independent posterior draws via predictive resampling.","lead":"The paper develops a new Bayesian regression method that uses recursive least squares to define a martingale posterior without requiring an explicit prior or Markov chain Monte Carlo. It allows the predictive distribution to be chosen separately from the mean estimator, enabling flexible modeling of heavy-tailed responses while keeping computation fast and order-invariant.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek/deepseek-v4-flash","headline":"The martingale property of the RLS coefficient sequence (Theorem 1) relies on the predictive distribution having mean exactly x^T β_{n-1}; this holds by construction under Assumption 1, but the proof of the martingale property (Appendix A.2) uses E(e_n | F_{n-1}, X_n) = 0, which is true because…","rationale":"The reader's verdict is ACCEPT with high confidence, and the weakest assumption identified is the convergence of predictive scales σ_n to a constant σ and the existence of the limiting second-moment matrix Ω. The paper acknowledges these assumptions and their limitations (Section 7). The central claim is well-supported by proofs and simulations. I agree with the reader's assessment. No load-bearing concern that would change the verdict.","tokens_in":36375,"tokens_out":899,"duration_ms":10723,"concrete_test":"Verify that the RLS update (3)-(4) indeed yields a martingale under the joint law P defined by (2) and Assumptions 1-2. This is already done in the proof of Theorem 1 (Appendix A.2). A concrete check: simulate data from a known linear model with Gaussian errors, compute β_n via RLS, and check that the sample mean of β̃ (obtained by predictive resampling with large N) equals β_n within Monte Carlo error. The paper does this in Section 5.1 (Table 1). The results confirm the theory. No additional test is needed.","verdict_should_be":"UNCHANGED","load_bearing_attack":"No significant objection identified. The paper constructs a martingale posterior for regression using recursive least squares. The central claim—that the RLS coefficient sequence is a martingale and converges to a random limit, establishing a martingale posterior with an order-invariant finite-sample posterior mean and asymptotic exchangeability—is supported by rigorous proofs (Theorems 1-3, Propositions 1-3). The assumptions (location-scale predictive family, convergence of predictive scales, existence of limiting second-moment matrix) are standard and stated transparently. The authors acknowledge the limitations: no GLM extension, non-robustness of the point estimate, no uncertainty propagation for hyperparameters, homoscedastic limiting model. The simulations and data analysis are consistent with the theory. I find no internal inconsistency or unsupported leap that would threaten the central argument. The paper is a solid contribution.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek/deepseek-v4-flash","summary":"The paper proposes a predictive Bayes framework for regression using recursive least squares (RLS). It shows that the RLS coefficient sequence is a martingale, converges to a random limit, and induces a martingale posterior. The posterior mean is order-invariant and available in closed form as a ridge-type estimator. The predictive distribution can be chosen separately (e.g., heavy-tailed) without changing the estimator. The paper derives exact posterior variance under fixed design, asymptotic Gaussian approximation, and closed-form martingale posterior for Gaussian predictive distributions. Extensions to additive models, mixed models, and Gaussian processes are provided. Hyperparameter selection via leave-one-out prequential log-likelihood is proposed. Simulations and an NHANES triglyceride data analysis demonstrate accuracy comparable to fully Bayesian models at lower computational cost.","tokens_in":36721,"tokens_out":2723,"duration_ms":20193,"significance":"The paper makes a significant contribution to predictive Bayes methodology by providing a flexible regression framework that avoids MCMC, yields independent posterior draws, and allows separate choice of predictive distribution. The theoretical results (Theorems 1-3, Propositions 1-3) are rigorous and clearly stated. The method addresses limitations of existing predictive Bayes approaches (order dependence, learning rate tuning). The extensions to additive, mixed, and GP models broaden applicability. The computational advantages are substantial. If the results hold, this could be a valuable tool for practitioners needing Bayesian-like uncertainty quantification with heavy-tailed or non-Gaussian responses in regression settings.","major_comments":[{"comment":"Assumption 2 requires the predictive scales σ_{n+1} = s_n(X_{n+1}) to converge P-almost surely to a constant σ for P_X-almost every x. This is a strong condition that may not hold in many practical settings, e.g., with heteroscedastic errors or non-stationary covariates. The authors acknowledge this limitation in §7 (\"the limiting model is homoscedastic\"), but the central asymptotic results (Theorem 1, Proposition 1, Theorem 2) all rely on this convergence. The paper would benefit from a discussion of how sensitive the results are to violations of this assumption, or whether the convergence can be relaxed to a weaker condition (e.g., convergence in probability or L^1).","section":"§3.1, Assumption 2","authors_acknowledge":"We therefore consider homoscedasticity a reasonable assumption for the response.","authors_acknowledge_p":0.74},{"comment":"Proposition 2 gives the exact posterior variance Σ_n = Σ_{i>n} σ_i^2 a_i (V_{i-1} - V_i), which depends on future covariates. This is a key result, but its practical utility is limited because future covariates are unknown. The authors note that with σ_n^2 = σ^2 / a_n, Σ_n = σ^2 V_n depends only on observed covariates. However, for general σ_n, the posterior variance is not available without specifying the future design. The paper should clarify that in practice, the default choice σ_n^2 = σ^2 / a_n is recommended, and that the general expression is mainly of theoretical interest. The simulation study in §5.1 uses both choices, but the default is clearly σ_n^2 = σ^2 / a_n.","section":"§3.2, Proposition 2","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"Algorithm 2 uses a shared covariate sequence across all B draws, which is a computational trick but may understate uncertainty about the covariate distribution. The authors claim in §5.1 that Algorithms 1 and 2 produce approximately the same posterior distribution, but this is only verified for the specific simulation settings (Gaussian linear model with n=3p,10p). It is not obvious that this equivalence holds more generally, e.g., with heavy-tailed predictive distributions or when n is small relative to p. The paper should either provide a theoretical justification or acknowledge this as a limitation that requires further study.","section":"§3.4, Algorithm 2","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The leave-one-out prequential log-likelihood (loo-pll) is proposed for hyperparameter selection. The authors claim it is invariant to ordering and more stable than the standard prequential log-likelihood. However, the justification that it is equivalent to averaging the prequential log-likelihood over all permutations (each observation comes last in (n-1)! of n! permutations) relies on the assumption that β_{-i} does not depend on the ordering of the other observations, which holds by Remark 1. This is correct, but the loo-pll still depends on the order of the other observations through the sequential construction? Actually, β_{-i} is computed from all observations except i, and by Remark 1 it is invariant to their ordering. So the loo-pll is indeed order-invariant. However, the loo-pll is not a proper scoring rule in the usual sense, and its use for hyperparameter selection may lead to overfitting. The paper should discuss potential overfitting or provide a reference justifying its use.","section":"§4.3, leave-one-out prequential log-likelihood","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"Table 2 compares rls-t, rls-n, blm, and BART. The blm (traditional Bayesian linear model with Student-t likelihood) uses MCMC with 5 chains of 2000 draws, burn-in 1000, giving ESS close to B=5000. However, the computational time for blm includes the full MCMC, while rls-t and rls-n times include only the posterior mean and predictive resampling. The comparison is somewhat unfair because blm provides full posterior inference (including uncertainty about hyperparameters), while rls-t fixes hyperparameters at their loo-pll estimates. The authors acknowledge this in §7 (\"the hyperparameters selected by the loo-pll are fixed in predictive resampling, so their uncertainty is not propagated\"). The paper should make this limitation more prominent when presenting the timing comparison.","section":"§5.3, Table 2","authors_acknowledge":"Fourth, the hyperparameters selected by theloo-pllare fixed in predictive resampling, so their uncertainty is not propagated, as shown by the equal widths forf (x) and Q0.90(x)in Table 2; a prior on them, as in Section 4.3, would address this.","authors_acknowledge_p":0.88},{"comment":"The paper lists several limitations: no GLM extension, non-robust point estimate, fixed p, no uncertainty propagation for hyperparameters, homoscedastic limiting model. These are appropriate and transparent. However, one additional limitation is that the method assumes the covariates are either iid from some distribution (random design) or fixed and bounded (fixed design). In many applications, covariates may be non-stationary or have complex dependence structures. The paper should discuss whether the method can accommodate such settings, or at least note this as a scope condition.","section":"§7, Limitations","authors_acknowledge":null,"authors_acknowledge_p":null}],"minor_comments":[{"comment":"The introduction mentions that existing parametric constructions rely on stochastic-gradient updates and have seen limited development for flexible regression. It would be helpful to cite specific references for these existing methods (e.g., Holmes and Walker 2023, Fong and Yiu 2026) earlier in the paragraph to ground the claim.","section":"§1, Introduction","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"Equation (5) gives β_n = V_n (V_0^{-1} β_0 + Σ_{i=1}^n y_i x_i). This is a standard result for recursive least squares. It might be helpful to note that this is equivalent to the solution of a penalized least squares problem with penalty matrix V_0^{-1}.","section":"§3.1, Equation (5)","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"Theorem 2 states that Σ_n^{-1/2} (β̃ - β_n) → N(0, I_p) conditionally on y_{1:n}. The proof uses the Lindeberg-Feller theorem. It would be helpful to explicitly state that the convergence is in distribution under P_{x_{1:∞}} (the fixed-design law), and that the result is conditional on the observed data.","section":"§3.3, Theorem 2","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The description of additive models uses V_0 = (Λ + ε I_r)^{-1} with ε → 0. This is a regularized inverse of the penalty matrix. It might be clearer to state that the posterior mean is the penalized least squares estimator with penalty matrix Λ, and that V_0 is chosen to approximate this.","section":"§4.1, Additive models","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"Table 1 reports log(||β̂ - β_n||^2 / p) and log||Σ̂ - Σ_n||_F. The values are negative, indicating small errors. It would be helpful to also report the actual MSE and Frobenius norm (not log) for interpretability, or at least note that the log scale is used for readability.","section":"§5.1, Table 1","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"Figure 1 shows the posterior distribution of a single regression coefficient. The caption should specify which coefficient is shown (the fourth, as mentioned in the text). Also, the figure legend uses \"Matching P_n\" and \"Gaussian P_n\" – it might be clearer to use \"Student-t P_n\" and \"Gaussian P_n\".","section":"§5.2, Figure 1","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The NHANES analysis uses a Student-t predictive distribution with ν=13.85 and φ=0.36. The paper should report the estimated smoothing parameters for the additive components, or at least note that they were selected by maximizing the loo-pll.","section":"§6, NHANES analysis","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"Lemma A.3 provides bounds on Σ_n. The proof uses the fact that Σ_{i>n} (V_{i-1} - V_i) = V_n. This is correct because the series telescopes and V_N → 0. However, the convergence of V_N to 0 is not explicitly stated in Assumption 3; it follows from the existence of Ω and the boundedness of covariates. The proof should note this.","section":"Appendix A, Lemma A.3","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"Table 3 reports results for Student-t responses. The column headers use \"apx\" for the Gaussian approximation N_p(β_n, Σ_n). It might be clearer to use \"approx\" or \"Gaussian approx\" to avoid confusion with the exact posterior.","section":"Appendix B.1, Table 3","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The pipeline for selecting the predictive distribution is useful but somewhat ad hoc. The paper should note that this is a heuristic and that more formal model selection criteria (e.g., WAIC, cross-validation) could be used.","section":"Appendix C, Pipeline","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The paper uses the notation P_n for the predictive distribution and P_ξ for the error distribution. This is clear, but the use of P for the joint law (e.g., in Theorem 1) might be confused with the predictive distribution. Consider using a different notation (e.g., ℙ) for the joint law.","section":"General","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The paper claims that posterior draws are independent by construction (Algorithm 2). This is true given the shared covariate sequence, but the draws are not independent across different covariate sequences. The paper should clarify that independence holds conditional on the imputed covariates.","section":"General","authors_acknowledge":null,"authors_acknowledge_p":null}],"recommendation":"major_revision"},"author_rebuttal":null,"desk_editor":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek/deepseek-v4-flash","headline":"Recursive least squares yields a martingale posterior for flexible regression","keywords":["martingale posterior","recursive least squares","predictive Bayes","Bayesian regression","flexible regression","additive models","leave-one-out prequential log-likelihood"],"falsifier":"If the RLS coefficient sequence β_n is not a martingale under the specified predictive distributions—for example, if the predictive mean is not x^T β_{n-1} or the predictive variance is not correctly specified—then Theorem 1 fails and the construction does not induce a martingale posterior. This could be tested by checking whether E(β_n | past) = β_{n-1} holds empirically in a simulation where the true data-generating process is known and the predictive distribution is misspecified.","tokens_in":37865,"feed_emoji":"📈","tokens_out":1344,"duration_ms":12781,"temperature":0.7,"pith_summary":"This paper shows that the recursive least squares (RLS) coefficient sequence is a martingale, so it induces a martingale posterior—an implicit prior and posterior on the regression coefficients without specifying a prior. The posterior mean is a ridge-type estimator that does not depend on data ordering, and the plug-in predictive distribution can be chosen separately (e.g., heavy-tailed) without changing the estimator. Under Gaussian predictive distributions, the exact martingale posterior is closed-form and matches the conjugate Bayesian linear model. This gives a fast, order-invariant Bayesian regression method that avoids MCMC, extends to additive, mixed, and Gaussian process models, and attains accuracy comparable to full Bayesian computation at a fraction of the cost.","feed_headline":"Recursive least squares yields a martingale posterior for regression","feed_subtitle":"A predictive Bayes framework that gives order-invariant posterior means and independent posterior draws, with accuracy comparable to MCMC…","key_machinery":"The recursive least squares (RLS) update (Equations 3-4), which defines the coefficient sequence β_n and matrix V_n. The key identity is that β_n is a martingale (Theorem 1), and the posterior distribution of the limit ˜β is a location family centered at β_n with independent increments from the predictive distribution. The series representation ˜β = β_n + Σ_{i>n} w_i ξ_i (Equation 6) is the central object: it separates the posterior mean (β_n) from the shape of the posterior (determined by the distribution of ξ_i).","core_discovery":"The paper establishes that the recursive least squares (RLS) update defines a martingale sequence of coefficient estimates, which converges to a random limit and thus induces a martingale posterior. The posterior mean is the RLS estimator, which is invariant to data ordering and coincides with a ridge regression estimator. The posterior uncertainty is characterized by a series representation: the random limit equals the RLS estimate plus a weighted sum of independent draws from the standardized predictive distribution. The predictive distribution can be chosen freely (e.g., Gaussian, Student-t) without altering the mean update. Under Gaussian predictive distributions and a specific predictive variance, the martingale posterior coincides exactly with the conjugate Bayesian linear model. The construction extends naturally to additive models, mixed models, and Gaussian process regression through appropriate choice of the initial matrix V0.","pith_inferences":["Because the posterior mean is always the ridge estimator regardless of the predictive distribution, the method is not robust to outliers in the mean: a heavy-tailed predictive distribution only affects posterior spread, not the point estimate. This is a fundamental limitation the authors acknowledge.","The asymptotic Gaussian approximation (Theorem 2) suggests that for large n, the martingale posterior is approximately normal with covariance σ²V_n, which depends only on observed covariates. This could enable approximate inference even without predictive resampling.","The series representation (Equation 6) implies that the martingale posterior is a location-scale family only when P_ξ is symmetric; for skewed P_ξ, the posterior inherits the skewness, which the simulation with skew Student-t errors confirms."],"forward_implications":["Posterior inference for regression coefficients becomes order-invariant and computationally cheap: the posterior mean is closed-form, and posterior draws are independent by construction, requiring no MCMC convergence diagnostics.","The predictive distribution can be chosen to match the response distribution (e.g., heavy-tailed for outlier-robust inference) without changing the estimator or requiring a new sampling algorithm.","The method extends to additive models, mixed models, and Gaussian process regression, providing a unified framework for flexible Bayesian regression with non-Gaussian responses.","Hyperparameters can be selected by maximizing the leave-one-out prequential log-likelihood, which is order-invariant and efficient, avoiding the instability of standard prequential likelihood."],"supporting_citations":[{"why":"Original reference for the recursive least squares update (Sherman-Morrison form).","marker":"Plackett (1950)"},{"why":"Provides the martingale posterior framework and predictive resampling method.","marker":"Fortini and Petrone (2020)"},{"why":"Develops parametric martingale posteriors via stochastic-gradient updates, which the present work extends to RLS.","marker":"Holmes and Walker (2023)"},{"why":"Introduces martingale posterior distributions and predictive resampling, providing the foundational methodology.","marker":"Fong et al. (2023)"},{"why":"Provides asymptotic theory for parametric martingale posteriors, used for comparison and extension.","marker":"Fong and Yiu (2026)"},{"why":"Discusses design distribution for predictive Bayesian regression; the RLS update is a special case of their score-based recursion under Gaussian P_ξ.","marker":"Sun and Fong (2026)"}],"fun_headline_variants":[],"cache_read_input_tokens":0,"weakest_assumption_plain":"The predictive distribution must belong to a location-scale family with known error distribution having zero mean and unit variance (Assumption 1), and the asymptotic results require the second-moment matrix of covariates to converge to a positive definite limit (Assumption 3) and the predictive scales to converge to a constant (Assumptions 2 and 3).","fun_headline_variants_meta":{"raw":{"variants":[]},"model":"deepseek/deepseek-v4-flash","effort":"low","cost_usd":9e-05,"raw_usage":{"cost":9.0181e-05,"is_byok":false,"cost_details":{"upstream_inference_cost":9.0181e-05,"upstream_inference_prompt_cost":8.9817e-05,"upstream_inference_completions_cost":3.64e-07},"total_tokens":989,"prompt_tokens":987,"completion_tokens":2,"prompt_tokens_details":{"audio_tokens":0,"video_tokens":0,"cached_tokens":0,"cache_write_tokens":0},"completion_tokens_details":{"audio_tokens":0,"image_tokens":0,"reasoning_tokens":0}},"tokens_in":987,"tokens_out":2,"cost_source":"openrouter_usage","duration_ms":2465,"image_pages":[],"temperature":1.0,"billed_attempts":1,"reasoning_tokens":0,"transport_provider":"openrouter","cache_read_input_tokens":0,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-11-13T03:09:10.240608+00:00","model_set":{"reader":"deepseek/deepseek-v4-flash"},"pipeline_version":"pith-pipeline@v1.4.1-daily-deepseek-or-medium","input_kind":null,"input_chars":null,"input_total_chars":null,"passes_missing":null,"published_standing":null,"published_confidence":null,"objection_audit":null,"run_record":null,"basis":{"kind":"not_recorded","source":null,"chars_given":null,"chars_total":null},"receipt":null,"falsifier":"If the RLS coefficient sequence β_n is not a martingale under the specified predictive distributions—for example, if the predictive mean is not x^T β_{n-1} or the predictive variance is not correctly specified—then Theorem 1 fails and the construction does not induce a martingale posterior. This could be tested by checking whether E(β_n | past) = β_{n-1} holds empirically in a simulation where the true data-generating process is known and the predictive distribution is misspecified.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Original reference for the recursive least squares update (Sherman-Morrison form)."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the martingale posterior framework and predictive resampling method."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Develops parametric martingale posteriors via stochastic-gradient updates, which the present work extends to RLS."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Introduces martingale posterior distributions and predictive resampling, providing the foundational methodology."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides asymptotic theory for parametric martingale posteriors, used for comparison and extension."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Discusses design distribution for predictive Bayesian regression; the RLS update is a special case of their score-based recursion under Gaussian P_ξ."}],"lane_accuracy":{"status":"not yet measured","lane":{"model":"deepseek/deepseek-v4-flash","input_kind":null},"url":"https://pith.science/accuracy"},"review_version":1}