REVIEW 3 major objections 4 minor 31 references
Bayesian Data Sketching for Varying Coefficient Regression Models
T0 review · 3 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read The paper claims that randomly compressing the data to $M \ll N$ rows via a Gaussian sketch still lets a Bayesian varying-coefficient model contract around the true functional coefficients at the rate $\theta_N = M_N^{-\xi/(2\xi+d)}$.
desk verdict The sketching idea is useful, but the contraction theorem rests on Assumption (E), which does not hold for the Gaussian sketch used. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The engine is the random compression matrix $\Phi$ with i.i.d. $N(0,1/N)$ entries, applied to the response $y$, the fixed-effect design $X$, and the basis-expanded varying-coefficient design $\tilde{X}B$, shrinking the sample from $N$ rows to $M_N$ rows. Two pieces carry the theory: a random-matrix result (Lemma 1, drawn from Vershynin 2010) bounding the eigenvalues of $\Phi\Phi^T$ near 1, which makes the compressed noise $\Phi\epsilon$ behave like $M$-dimensional i.i.d. noise; and Assumption (E), which asserts that the compressed design $\tilde{X}_{\Phi,N}$ preserves the spectral norm of the uncompressed design up to a factor $\kappa_N$ satisfying $1 \prec N\kappa_N \prec M_N$, so that divergence and prior-mass calculations reduce to standard B-spline approximation theory. The basis expansion of each varying coefficient into B-splines or similar bases is what converts the functional problem into a linear regression on basis coefficients $\gamma$, which is then analyzed on the compressed data.
What would settle it
Take the paper's own setting—Gaussian $\Phi$, bounded covariates, B-spline basis—and compute, over many draws of $\Phi$ and many coefficient vectors $\alpha$, the ratio $\|\tilde{X}_{\Phi,N}\alpha\|_2 / \|\tilde{X}_N\alpha\|_2$. If the ratio fails to concentrate around a single $\kappa_N$ satisfying $1 \prec N\kappa_N \prec M_N$ as $N$ and $M_N$ grow, then Assumption (E) fails and Theorem 4's rate does not follow. A simpler empirical check is to simulate a true $w^*$ of known smoothness $\xi$, fit the compressed model with two different growth rates for $M_N$, and test whether the integrated $L^2$ error of the estimated varying coefficients decays like $M_N^{-\xi/(2\xi+d)}$; a systematic departure from that power law would contradict the claimed contraction rate.
Extended reading notes
Core claim
The central claim is that compression does not destroy Bayesian learning for varying-coefficient models: under regularity conditions—bounded covariates, smoothness $\xi$, basis approximation error $O(H_N^{-\xi})$, and a spectral-preservation condition on $\Phi$—the posterior from the compressed model contracts around the true varying coefficients $w^*$ at rate $\theta_N \asymp M_N^{-\xi/(2\xi+d)}$ as both $N$ and the compressed dimension $M_N$ grow, and the plug-in predictive densities converge to the truth in Hellinger distance. Because this is the minimax-optimal rate for estimating $\xi$-smooth functions, the compressed model is presented as an asymptotically valid surrogate: a user who never touches the uncompressed data can still obtain consistent, adaptive, fully model-based Bayesian inference. The proof bounds the Kullback–Leibler divergence between the compressed and true likelihoods, uses random-matrix spectral bounds to show that $\Phi\Phi^T$ is close to the identity, and uses a key assumption to transfer spectral properties of the compressed design matrix to the uncompressed one.
Load-bearing premise
The load-bearing premise is Assumption (E): a random Gaussian projection preserves the spectral geometry of the design matrix closely enough that the compressed design's norm on every coefficient vector is proportional to the uncompressed norm with a slowly growing factor $\kappa_N$, where $1 \ll N\kappa_N \ll M_N$; the paper calls this reasonable for Gaussian sketches but does not prove it for the actual sketch used, and the contraction-rate proof collapses without it.
Editorial extensions
If this is right
- A user can take any existing Bayesian varying-coefficient sampler, apply it to the sketched data $(y_\Phi, X_\Phi, \tilde{X}_\Phi)$, and obtain posterior uncertainty statements that remain valid as the sample size grows, without ever fitting or storing the uncompressed data.
- The contraction rate $\theta_N = M_N^{-\xi/(2\xi+d)}$ depends only on the compressed dimension, so the accuracy versus computation trade-off is controlled directly through $M_N$, with $M_N = o(N)$ required and $M_N \approx 10\sqrt{N}$ used in the paper's experiments.
- The predictive density at a new location converges in Hellinger distance to the true predictive density, so predictions and predictive intervals from the compressed model are asymptotically calibrated.
- The basis rank $H_N$ and compressed dimension $M_N$ must grow together ($H_N \asymp M_N^{1/(2\xi+d)}$), so more complex functional surfaces require both more basis functions and more sketch rows.
Reading between the lines
- A natural next test is whether Assumption (E), the spectral-preservation condition, actually holds for the Gaussian sketch used in the paper; the authors cite general sketching properties rather than verifying it directly, so a dedicated calculation for the compressed design would either close the gap or reveal a restricted regime.
- The same proof template suggests that sparser sketches such as Hadamard or Clarkson–Woodruff transforms should also work if they satisfy (E); confirming this empirically could extend the speed advantage to settings where Gaussian sketching's $O(MN^2\tilde{P})$ cost is itself a bottleneck.
- The theory fixes $\sigma^2 = 1$ and $\beta = 0$; extending the results to unknown error variance and nonzero fixed effects would remove two simplifications the authors flag, and the Hellinger predictive result suggests the extension should go through with a bounded-support prior on $\sigma^2$.
- The paper conditions on the realized sketch $\Phi$; averaging inference over repeated independent sketches would produce credible intervals that reflect sketching randomness rather than a single draw, a modification the current theory does not cover.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper develops a Bayesian data sketching framework for varying coefficient regression models. The response and predictor matrices are compressed by an M_N x N Gaussian random matrix, and a Bayesian VCM with basis-expanded varying coefficients is fitted to the compressed data. The main theoretical claims are a posterior contraction rate theta_N = M_N^{-xi/(2xi+d)} for the varying coefficients and Hellinger consistency of the posterior predictive density, established under Assumptions (A)-(F). The paper also gives MCMC algorithms, simulation comparisons against the uncompressed model and a predictive-process sparse GP, and an application to MODIS NDVI data. The central thesis is that the compressed model is an asymptotically valid surrogate, so existing algorithms for uncompressed VCMs can be applied to the compressed data without developing new model classes or specialized hardware.
Significance. If the contraction result were established for the sketching matrices actually used, the paper would make a useful practical contribution: it would provide a principled, algorithm-agnostic way to obtain Bayesian inference for massive functional data by compressing to M ~ sqrt(N) observations, with explicit MCMC implementations and reproducible code. The empirical sections show that the method gives predictive coverage close to nominal and substantial computational savings relative to the uncompressed model. The theoretical framework combining random-matrix bounds with nonparametric posterior contraction is a natural extension of earlier compressed-regression work. However, the paper's two most load-bearing assumptions about the sketching matrix are either false as stated or not satisfied by the Gaussian sketch, so the main theorem does not currently cover the implemented method; the proofs are likely repairable by a subspace-embedding reformulation, but the repair is substantive.
major comments (3)
- [Section 3.2, Assumption (E); Appendix A and B] Assumption (E) is false for the Gaussian sketch used throughout the paper. In the spatially varying intercept case with tilde P = 1 and tilde x_1(u) = 1, the matrix tilde X_N equals I_N, so (E) requires ||Phi alpha||_2 approx kappa_N ||alpha||_2 for every alpha in R^N. Since Phi is M_N x N with M_N < N, it has a nontrivial null space; for any nonzero v in that null space the left-hand side is zero while the right-hand side is positive, a contradiction. For tilde P > 1 with an intercept term, the map alpha -> tilde X_N alpha is onto R^N for generic covariates, so the same obstruction occurs. Because Lemmas 2 and 3 invoke (E) directly, the contraction rate in Theorem 4 is not established for the Gaussian sketch. The proof actually needs only a restricted equivalence on the H_N tilde P-dimensional subspace spanned by the columns of tilde X_N B, together with an upper bound for the approximation-error term; such a subspace embedding is plausible for Gaussian sketches. The assumption must be restated and proved in that restricted form, and the condition 1 << N kappa_N << M_N must be reconciled with the natural value kappa_N = M_N/N, for which N kappa_N equals M_N rather than being o(M_N).
- [Section 3.2, Assumption (C); Appendix A] Assumption (C) is not satisfied by the Gaussian sketch with the stated order. For Phi with independent N(0,1/N) entries, the squared Frobenius norm ||Phi Phi^T - I||_F^2 is of order M_N^2/N: the M_N diagonal terms contribute O(M_N/N) and the M_N(M_N-1) off-diagonal terms contribute O(M_N^2/N). Thus ||Phi Phi^T - I||_F is of order M_N / sqrt(N), not sqrt(M_N/N). Consequently the display after equation (16), which bounds sum_k (1-e_k)^2 by ||I - Phi Phi^T||_F^2 << M_N/N, understates the left-hand side by a factor of M_N. The cited Vershynin results give an operator-norm bound, not the Frobenius bound used here. The proof of Lemma 2 may still be repairable under the paper's empirical scaling M_N ~ sqrt(N), but Assumption (C) and the eigenvalue-sum bound need to be corrected.
- [Section 4.1, Table 1] The claim that geoS 'outperforms the uncompressed model' should be restricted to computational efficiency. In every scenario in Table 1 the compressed model has larger MSE for the varying coefficients, larger credible-interval lengths, and larger predictive-interval lengths than the uncompressed model; for example, at N = 5000 the MSE is 0.0335 versus 0.0109. The method does deliver nearly nominal coverage and substantially better computational efficiency, which is the relevant claim, but the current wording overstates the inferential comparison.
minor comments (4)
- [Section 2.1] The stated cost O(M N^2 tilde P) for constructing the Gaussian sketch overstates the cost: because tilde X is block diagonal with tilde P nonzeros per row, the multiplication Phi tilde X costs O(M N tilde P), not O(M N^2 tilde P), unless tilde X is treated as dense.
- [Section 3.2] The text says Assumptions (C) and (E) 'can be proved to hold with high probability' and cites Ahfock et al. (2017), but no proof or precise reference to a theorem is given for (E). Given that (E) as written is false, this sentence is misleading and should be revised to state exactly which restricted property is being assumed.
- [Section 4.2] The choice M ~ 10 sqrt(N) is justified only by empirical stabilization in Figure 3; the paper should state explicitly that this is a heuristic and is not derived from the conditions of Theorem 4.
- [Appendix C] In the proof of Theorem 4, the sentence beginning 'The second expression in the second line follows from Lemma A.1' refers to the third displayed line, not the second; the cross-reference is confusing.
Circularity Check
No significant circularity: the posterior contraction theorem is proved from stated assumptions, and the self-citations are not load-bearing.
full rationale
The paper's central claim is that the posterior from the compressed model (4) contracts around the true varying coefficients at rate M_N^{-xi/(2xi+d)} (Theorem 4), with predictive Hellinger consistency in Theorem 5. The proof derives this from explicit assumptions (A)-(F), building on random-matrix bounds from Vershynin and standard Bayesian contraction techniques from Ghosal et al., Jeong and Ghosal, and Laurent and Massart. These are external, machine-checkable-style mathematical ingredients, not fitted quantities. Assumption (E) is genuinely an assumption, not a definition of the conclusion; the paper does not fit a parameter to data and then rename it a prediction. The empirical choice M ~ 10*sqrt(N) is a heuristic, but it is not presented as a derived rate or as forced by the theory. The self-citations to Guhaniyogi and Dunson (2015, 2016) and Guhaniyogi and Scheffler (2021) are used to motivate the general sketching framework and to note that treating random-matrix assumptions as holding with high probability is common practice; they do not supply the proof of Theorem 4. The possible failure of Assumption (E) for the Gaussian sketch, as highlighted in the skeptical reading, is a correctness or applicability concern about an explicit assumption, not a circularity: the theorem is conditional on (E), and no circular reduction between the theorem's conclusion and its inputs is present. The derivation is self-contained relative to its stated assumptions, so the circularity score is 0.
Assumptions & free parameters
free parameters (2)
- M (compressed dimension) =
M = 10*sqrt(N) in experiments; theoretical M_N grows with N
- Number of basis functions H =
H = 225, 256, 400 in simulations; H = 1521 in real data
assumptions (4)
- domain assumption The true varying coefficients w*_j lie in the smoothness class F_xi(D) of xi-times continuously differentiable functions, and basis approximation error is O(H^{-xi}) (Assumption A)
- domain assumption The random compression matrix Phi satisfies ||Phi*Phi^T - I||_F <= C' sqrt(M/N) and, crucially, Assumption (E) on the restricted isometry-like behavior of Phi w.r.t. X_tilde_N
- domain assumption The covariates x_p(u) are uniformly bounded by 1 (Assumption D)
- ad hoc to paper beta = 0, sigma^2 = 1 known, and Delta = I in the theoretical analysis (Assumption F)
Cite this review
Pith. "Pith review of Bayesian Data Sketching for Varying Coefficient Regression Models." pith.science (2026). https://pith.science/paper/UPPQQL55
@misc{pith2026250600270,
author = {Pith},
title = {Pith review of: Bayesian Data Sketching for Varying Coefficient Regression Models},
year = {2026},
howpublished = {\url{https://pith.science/paper/UPPQQL55}},
note = {Machine review of arXiv:2506.00270}
}
read the original abstract
Varying coefficient models are popular for estimating nonlinear regression functions in functional data models. Their Bayesian variants have received limited attention in large data applications, primarily due to prohibitively slow posterior computations using Markov chain Monte Carlo (MCMC) algorithms. We introduce Bayesian data sketching for varying coefficient models to obviate computational challenges presented by large sample sizes. To address the challenges of analyzing large data, we compress the functional response vector and predictor matrix by a random linear transformation to achieve dimension reduction and conduct inference on the compressed data. Our approach distinguishes itself from several existing methods for analyzing large functional data in that it requires neither the development of new models or algorithms, nor any specialized computational hardware while delivering fully model-based Bayesian inference. Well-established methods and algorithms for varying coefficient regression models can be applied to the compressed data.
Figures
Reference graph
Works this paper leans on
-
[1]
Statistical properties of sketching algorithms
D. Ahfock, W. J. Astle, and S. Richardson. Statistical properties of sketching algorithms. arXiv preprint arXiv:1706.03665 ,
-
[6]
doi: 10.1145/3019134. N. Cressie and G. Johannesson. Fixed rank kriging for very large spatial data sets. Journal of the Royal Statistical Society: Series B (Statistical Methodology) , 70(1):209–226,
-
[11]
doi: https://doi.org/10.1016/j.envsoft.2019.104608
ISSN 1364-8152. doi: https://doi.org/10.1016/j.envsoft.2019.104608. URL https://www.sciencedirect. com/science/article/pii/S1364815219310412. A. O. Finley, S. Banerjee, and D. W. MacFarlane. A hierarchical model for quantifying forest variables over large heterogeneous landscapes with uncertain forest areas. Journal of the American Statistical Association...
-
[12]
doi: 10.1080/00401706. 2018.1437474. URL https://doi.org/10.1080/00401706.2018.1437474. PMID: 31007296. R. Guhaniyogi and S. Banerjee. Multivariate spatial meta kriging. Statistics & probability letters, 144:3–8,
-
[13]
Large Multi-scale Spatial Kriging Using Tree Shrinkage Priors
R. Guhaniyogi and B. Sans´ o. Large multi-scale spatial kriging using tree shrinkage priors. arXiv preprint arXiv:1803.11331 ,
-
[14]
Sketching in Bayesian High Dimensional Regression With Big Data Using Gaussian Scale Mixture Priors
R. Guhaniyogi and A. Scheffler. Sketching in Bayesian high dimensional regression with big data using gaussian scale mixture priors. arXiv preprint arXiv:2105.04795 ,
-
[18]
doi: 10.1080/01621459.2015. 1123632. URL http://dx.doi.org/10.1080/01621459.2015.1123632. M. Katzfuss and J. Guinness. A General Framework for Vecchia Approximations of Gaussian Processes. Statistical Science , 36(1):124 – 141,
arXiv 2015
-
[20]
URL https://doi.org/10.1080/10618600.2020.1754225
doi: 10.1080/10618600.2020.1754225. URL https://doi.org/10.1080/10618600.2020.1754225. A. Korattikara, Y. Chen, and M. Welling. Austerity in mcmc land: Cutting the metropolis- hastings budget. In International conference on machine learning, pages 181–189. PMLR,
arXiv 2020
Show all 31 references
- [21]
-
[22]
M. W. Mahoney. Randomized algorithms for matrices and data. arXiv preprint arXiv:1104.5557,
-
[23]
URL http://dx.doi.org/10.1080/10618600.2014.914946
doi: 10.1080/10618600.2014.914946. URL http://dx.doi.org/10.1080/10618600.2014.914946. M. Peruzzi, S. Banerjee, and A. O. Finley. Highly scalable Bayesian geostatistical modeling via meshed gaussian processes on partitioned domains.Journal of the American Statistical Associati...
2014
-
[25]
ISBN 0769527205
IEEE Computer Society. ISBN 0769527205. doi: 10.1109/FOCS.2006.37. URL https://doi.org/10.1109/FOCS.2006.37. D. Seita, X. Pan, H. Chen, and J. Canny. An efficient minibatch acceptance test for metropolis-hastings. arXiv preprint arXiv:1610.06848 ,
2006 arXiv
-
[27]
Vershynin
28 Bayesian Data Sketching R. Vershynin. Introduction to the non-asymptotic analysis of random matrices. arXiv preprint arXiv:1011.3027,
-
[29]
X. Yuan, P. Llull, D. J. Brady, and L. Carin. Tree-structure bayesian compressive sensing for video. arXiv preprint arXiv:1410.3080 ,
-
[1996]
URL https: //onlinelibrary.wiley.com/doi/abs/10.1111/j.1538-4632.1996.tb00936.x
doi: https://doi.org/10.1111/j.1538-4632.1996.tb00936.x. URL https: //onlinelibrary.wiley.com/doi/abs/10.1111/j.1538-4632.1996.tb00936.x. D. R. Burt, C. E. Rasmussen, and M. van der Wilk. Convergence of sparse variational inference in gaussian processes regression. arXiv prepr...
1996
-
[2004]
Z. Huang. Near optimal frequent directions for sketching dense and sparse matrices. In International Conference on Machine Learning , pages 2048–2057. PMLR,
-
[2005]
A. v. d. Vaart and H. v. Zanten. Information rates of nonparametric gaussian process methods. Journal of Machine Learning Research , 12(Jun):2095–2119,
-
[2006]
URL https://onlinelibrary
doi: https://doi.org/10.1111/j.1541-0420.2006.00535.x. URL https://onlinelibrary. wiley.com/doi/abs/10.1111/j.1541-0420.2006.00535.x. S. Banerjee, A. E. Gelfand, A. O. Finley, and H. Sang. Gaussian predictive process models for large spatial data sets. Journal of the Royal Sta...
2006
-
[2009]
doi: 10.1137/060673096. R. Bai, M. R. Boland, and Y. Chen. Fast algorithms and theory for high-dimensional Bayesian varying coefficient models. arXiv preprint arXiv:1907.06477 ,
1907 arXiv
-
[2010]
E., Diggle, P., Fuentes, M
Gelfand, A. E., Diggle, P., Fuentes, M. and Guttorp, P., editors, Chapman and Hall/CRC, pp. 107-118. D. P. Woodruff. Sketching as a tool for numerical linear algebra. arXiv preprint arXiv:1411.4357,
-
[2011]
doi: https://doi.org/10.1111/j.2041-210X.2010. 00060.x. URL https://besjournals.onlinelibrary.wiley.com/doi/abs/10.1111/ j.2041-210X.2010.00060.x. 25 Guhaniyogi, Baracaldo and Banerjee A. O. Finley and S. Banerjee. Bayesian spatially varying coefficient models in the spbayes r...
2010 arXiv
-
[2013]
Guhaniyogi, C
R. Guhaniyogi, C. Li, T. D. Savitsky, and S. Srivastava. A divide-and-conquer Bayesian approach to large-scale kriging. arXiv preprint arXiv:1712.09767 , 2020a. R. Guhaniyogi, C. Li, T. D. Savitsky, and S. Srivastava. Distributed Bayesian varying coefficient modeling using a g...
2006 arXiv
-
[2015]
Jeong and S
S. Jeong and S. Ghosal. Unified bayesian asymptotic theory for sparse linear regression. arXiv preprint arXiv:2008.10230 ,
2008 arXiv
-
[2016]
2015.1044091
URL http://dx.doi.org/10.1080/01621459. 2015.1044091. C. De Sa, V. Chen, and W. Wong. Minibatch gibbs sampling on large graphical models. In International Conference on Machine Learning , pages 1173–1181,
2015
-
[2017]
23 Guhaniyogi, Baracaldo and Banerjee S. Ahn, A. Korattikara, and M. Welling. Bayesian posterior sampling via stochastic gradient fisher scoring. arXiv preprint arXiv:1206.6380 ,
-
[2018]
S. K. Deshpande, R. Bai, C. Balocchi, J. E. Starling, and J. Weiss. Vcbart: Bayesian trees for varying coefficients. arXiv preprint arXiv:2003.06416 ,
2003 arXiv
-
[2019]
URL https://onlinelibrary.wiley.com/doi/abs/10.1002/sam.11413
doi: https://doi.org/10.1002/ sam.11413. URL https://onlinelibrary.wiley.com/doi/abs/10.1002/sam.11413. L. Zhang, W. Tang, and S. Banerjee. Bayesian geostatistics using predictive stacking,
-
[2020]
Dobriban and S
E. Dobriban and S. Liu. A new theory for sketching in linear regression. arXiv preprint arXiv:1810.06089,
-
[2021]
URL https://doi.org/10.1214/19-STS755
doi: 10.1214/19-STS755. URL https://doi.org/10.1214/19-STS755. M. Kim and L. Wang. Generalized spatially varying coefficient models. Journal of Compu- tational and Graphical Statistics , 30(1):1–10,
-
[2022]
URL https: //doi.org/10.1080/01621459.2020.1833889
doi: 10.1080/01621459.2020.1833889. URL https: //doi.org/10.1080/01621459.2020.1833889. T. Sarlos. Improved approximation algorithms for large matrices via random projections. In Proceedings of the 47th Annual IEEE Symposium on Foundations of Computer Science , FOCS ’06, page ...
2020
-
[2024]
URL https://arxiv.org/abs/2304.12414. S. Zhou, L. Wasserman, and J. D. Lafferty. Compressed regression. In Advances in Neural Information Processing Systems, pages 1713–1720,
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.