{"id":"9959b2e1-45bb-4d62-8dcc-c5878072d76f","arxiv_id":"2411.17808","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"The spar R package provides an extensible ensemble implementation of screened, randomly projected generalized linear models for high-dimensional data.","lead":"This paper introduces an R package that fits many small regression models on randomly screened and projected subsets of predictors, then averages them for high-dimensional data. The package makes a recent ensemble method easy to use and extend with custom screening and projection procedures.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"spar.cv's default cross-validation leaks validation information into screening: inds and RPMs are generated on the full data and reused in every fold, violating the CV protocol of Section 2.4. This can make reported CV measures optimistically biased and selected (M, nu) unreliable.","rationale":"The reader's weakest assumption (recovery property of rp_cw) is plausible but not the most immediate threat to the package's central claim. The paper's own text of Section 3.1 explicitly describes a CV procedure that reuses full-data screening indices in every fold; this contradicts Section 2.4's CV protocol and is verifiable without external theory. The CV leakage directly affects the reliability of model selection, which is a core advertised functionality (spar.cv). It is concrete, internal, and fixable. The recovery property, while important, is inherited from a cited technical report and would only affect the flagship projection mode; the package can still function with data-agnostic projections. Therefore I flag the CV leakage as the single load-bearing concern. Agreement with reader: disagree, because the reader identified a different weakest assumption. Verdict remains CONDITIONAL: the authors should fix or clearly document the leakage before the package's CV results are taken at face value.","tokens_in":1,"tokens_out":6446,"duration_ms":119814,"concrete_test":"Simulate n=100, p=500, y independent of X (all beta=0). Run spar.cv(..., nfolds=5, nummods=c(5,20), measure='mse') with default screencoef=screen_glmnet and record the CV measure. Then implement a manual leakage-free CV: for each fold, call spar() on the training portion with screencoef recomputed on that training portion (and projection updated), evaluate on the validation portion. If the default spar.cv's average CV measure is systematically lower (better) than the manual CV's, the fixed full-data screening indices are leaking validation information. Repeat with 50 independent noise datasets to get a distribution of the difference; if the default beats the honest CV by more than the standard error, the leakage is real and material.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The most load-bearing weakness is not the inherited recovery property of rp_cw but the implementation of spar.cv. Section 3.1 states that spar.cv calls spar() in each fold with 'predefined inds and RPMs ... generated by first calling spar() on the whole data set'. The SPAR algorithm in Section 2.4 includes screening as an internal step: Step 3 computes screening coefficients and Step 4.1 selects the screened index set. Step 6 says CV should 'repeat steps 1 to 4', so a valid CV must recompute screening coefficients and index sets on each training fold. By fixing inds from the full data, the validation folds' responses influence which predictors enter every training-fold model. This is a form of selection leakage: the CV measure K(M, nu) is not an unbiased estimate of generalization error for the full screening-then-projection pipeline. The optional update_rpm_w_data() only refreshes data-driven projection entries in RPMs; it does not regenerate the screening indices, so the leakage remains for the default screencoef. Since spar.cv is the package's only model-selection tool and is used in Sections 5.1 and 5.2, the reported CV-optimal (M, nu) and the face-image '1se' sparsity choice may be optimistic. The paper does not warn users of this limitation or offer a leakage-free mode.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper describes the R package \"spar\", which implements sparse projected averaged regression (SPAR) for high-dimensional generalized linear models. The algorithm combines marginal screening, random projection, per-model penalized GLM fitting, thresholding, and averaging. The package provides S3 classes for screening coefficients and random projections, including a data-driven projection (rp_cw) that is claimed to approximately preserve the true coefficient vector. The paper gives an algorithmic description, package documentation, two extensibility examples, and two real-data illustrations on the Isomap faces data and the Darwin Alzheimer's handwriting data.","tokens_in":20897,"tokens_out":5786,"duration_ms":53766,"significance":"If the implementation is correct, the package would be a useful contribution because it offers a flexible, extensible framework for combining screening and random projection in GLM ensembles, and the S3 class design is a genuine strength that lowers the barrier for users to plug in custom procedures. The paper also gives credit to the source algorithms and clearly attributes the data-driven projection to earlier technical reports by the same authors. However, the paper currently lacks any independent empirical verification of the algorithm's recovery or predictive properties, and the cross-validation implementation as described has a selection-leakage problem that undermines the main model-selection tool. These issues are fixable, but they are load-bearing for the paper's central claims.","major_comments":[{"comment":"The cross-validation procedure in spar.cv leaks validation information into the screening step. Section 3.1 states that spar.cv generates inds and RPMs by first calling spar() on the whole data set, and then passes these predefined objects to spar() in each fold. However, Algorithm 2.4 specifies that screening is an internal step: Step 3 computes screening coefficients and Step 4.1 selects the screened index set from the data. Reusing full-data screening indices in every fold means the validation-fold responses influence which predictors enter each training-fold model, so the cross-validated measure K(M, nu) is not an unbiased estimate of generalization error for the full screening-then-projection pipeline. The optional update_rpm_w_data() refreshes only the data-dependent entries in the projection matrices; it does not regenerate the screening indices. This is particularly serious because spar.cv is the package's only model-selection tool and is used in Sections 5.1 and 5.2 to choose (M, nu). The paper should either recompute screening coefficients and indices on each training fold inside the CV loop, or explicitly document and justify this as an approximate CV scheme, with a simulation demonstrating that the leakage has negligible effect on the selected parameters.","section":"Section 3.3 and Section 6"},{"comment":"The paper's default projection rp_cw(data=TRUE) is repeatedly claimed to approximately capture the true regression coefficients in the span of the projection, and Section 6 asserts that SPAR with this projection performs effectively across different degrees of sparsity. These claims are supported only by citations to the authors' technical reports (Parzer et al. 2024a,b); the manuscript itself contains no simulation with known coefficients, no comparison with existing high-dimensional estimators, and no runtime benchmark. Because the recovery property is the main justification for the default projection and for the interpretability of the resulting coefficients, the paper should include at least one synthetic-data experiment that checks coefficient recovery under the implemented rp_cw(data=TRUE), and ideally a small comparison against a baseline such as glmnet to demonstrate that the implementation reproduces the expected prediction behavior. Without such verification, a reader cannot distinguish implementation errors from expected algorithm behavior.","section":"Section 3.3, Section 6, and Section 5"}],"minor_comments":[{"comment":"The caption says the plot uses a fixed optimal number of models M = 20, but the text and the printed output for the faces example report the smallest CV measure at nummod=10. The caption should be corrected to M = 10.","section":"Figure 1"},{"comment":"In the two function signatures, the default for the measure argument is shown as c(\"deviance\", \"mse\", \"mae\", \"class\", \"1-auc\"), while the explanatory text says the default is \"deviance\". This inconsistency should be fixed so that the documentation matches the actual R default.","section":"Section 3.1"},{"comment":"The equation for the marginal likelihood screening coefficient reads \"ˆ\\omega_j =: argmin\" but the notation appears to be reversed; it should be \"\\hat\\omega_j := \\argmin\" or simply \"\\hat\\omega_j = \\argmin\".","section":"Section 2.1"},{"comment":"The conclusion contains the sentence \"The package provides an implementation an algorithm\" which is missing a word; it should read \"an implementation of an algorithm\".","section":"Section 6"}],"recommendation":"major_revision","confidential_remarks":"The CV leakage issue in spar.cv is the decisive technical problem; it is a fixable implementation flaw but it directly affects the validity of the reported cross-validated results. I would not reject on the basis of reliance on the authors' technical reports, but the paper would be much stronger with a synthetic-data verification of the recovery property and a small comparison study. The S3 design is a genuine strength and the examples are helpful, so the paper is worth a revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First thing to know: this is a software paper, and the software appears to be real and competently built. The genuinely new deliverable is the spar package—the first R implementation of the SPAR ensemble for GLMs, with an S3 class system that lets users plug in custom screening coefficients and random projections. That extensibility is the best part of the paper: the examples in Section 4 show a VariableScreening SIRS coefficient and a Cannings–Samworth projection dropping in cleanly, and the code listings are consistent with the algorithm in Section 2.4. The two real-data illustrations (faces, Darwin) are reproducible from the text and give a fair sense of how the package behaves.\n\nThe soft spots are not fatal, but they are real. Most importantly, the default spar.cv procedure generates screening indices (inds) and random projection matrices (RPMs) once on the full data, then reuses them in every fold. The SPAR algorithm's own cross-validation step says to repeat steps 1–4, which includes screening. Fixing the screening set from the full data means the validation-fold responses influence which predictors enter every training model. That is selection leakage, and it makes the reported CV measures optimistic and the chosen (M, nu) unreliable. The paper states this design choice openly but does not warn about the consequence, and update_rpm_w_data only refreshes projection entries, not screening indices. This is the main thing I would want changed before trusting the package's model selection.\n\nTwo smaller issues. The paper contains no comparison against ordinary high-dimensional baselines like glmnet, no runtime benchmarks, and no package version or commit pin, so the reader cannot reproduce the exact state of the code used. And the central recovery claim for rp_cw(data=TRUE) is inherited from the authors' own technical reports; the paper just cites it. For a software paper that is acceptable, but it means the performance claims rest on unpublished-in-this-venue results.\n\nWho is this for? Applied statisticians working in p>>n settings who want a flexible, interpretable GLM ensemble and are willing to install from GitHub. It deserves a serious referee: the artifact is real, the design is thoughtful, and the CV leakage is fixable. I would send it out with a request to address the leakage and add a small benchmark.","headline":"Solid software paper for an existing ensemble method; the package looks real and well-designed, but the default CV procedure leaks validation information into screening and needs fixing or a warning before the reported results are taken at face value.","tokens_in":21447,"tokens_out":2164,"would_cite":true,"duration_ms":18612,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["62-04","62J07","62J12"],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper presents the R package spar, the first implementation of sparse projected averaged regression for generalized linear models, together with an S3-class framework that lets users plug in custom screening and projection procedures.","keywords":["random projection","variable screening","ensemble learning","generalized linear models","high-dimensional regression","sparse projected averaged regression","R package","S3 classes"],"falsifier":"Run spar() on the package's included simulated data, which explicitly provides the true coefficient vector, and compare rp_cw(data = TRUE) with a data-agnostic projection such as rp_gaussian(): if the data-driven default does not produce closer coefficient recovery or better variable ranking, the approximate-span premise is unsupported. A second check is to search the package ecosystem for an existing GLM ensemble that already performs screening, random projection, and averaging, which would directly test the first-implementation claim.","tokens_in":20333,"feed_emoji":"📦","tokens_out":12354,"duration_ms":96993,"temperature":0.7,"pith_summary":"This paper presents spar, an R package that implements sparse projected averaged regression (SPAR) for generalized linear models when predictors vastly outnumber observations. Each member of the ensemble screens the predictor set, projects the survivors to a random low-dimensional space, fits a penalized GLM, and thresholds the resulting coefficients, then the members are averaged back in the original variable space. The package's central claims are first implementation and extensibility: the screening, projection, and marginal-model steps are all S3 classes, so users can supply their own procedures through constructor functions. The default projection is data-driven, replacing random diagonal entries of a sparse embedding with small-penalty ridge coefficients, which the authors say keeps the true coefficients approximately recoverable after projection. The practical payoff would be computationally feasible high-dimensional GLM fitting whose coefficients remain interpretable in terms of the original predictors.","feed_headline":"spar: first R package for sparse projected averaged GLM ensembles","feed_subtitle":"Screening plus data-driven random projection lets GLM ensembles scale to p >> n without losing coefficient meaning.","key_machinery":"The load-bearing object is the SPAR ensemble loop: per-model screening, random projection, penalized GLM fit, coefficient thresholding, and averaging. The identity that gives the default its interpretability is the data-driven sparse embedding rp_cw(data = TRUE), where random diagonal entries are swapped for minimal-penalty ridge coefficients so the true beta is claimed to lie approximately in the projection's span. Around these sit the three S3 classes—'screencoef' for screening coefficients, 'randomprojection' for projection matrices, and 'sparmodel' for marginal models—each exposing a generate_fun component that the main spar() and spar.cv() functions invoke, which is what makes any of the steps replaceable.","core_discovery":"On its own terms, the paper claims that spar is the first implementation of the SPAR algorithm for GLMs, and that its S3 architecture delivers genuine extensibility. The algorithm works model by model: screen to at most 2n predictors based on a screening coefficient, project the selected variables to a random dimension chosen between log(p) and n/2, estimate a (possibly L2-penalized) GLM on the projected data, map the coefficients back to original variables, and threshold small entries before averaging across M models. The distinctive default is rp_cw(data = TRUE), a Clarkson-Woodruff-style sparse embedding whose random diagonal entries are replaced by ridge coefficients from a minimal penalty; the paper states this construction approximately captures the true regression coefficients in the span of the projection. The paper demonstrates extensibility by wrapping a SIRS screening coefficient from another package and by building a Cannings-Samworth-style random projection that selects among Haar matrices by validation error.","pith_inferences":["Beyond the paper, the same S3 machinery could house non-GLM base learners such as Cox proportional hazards or quantile regression by defining a new marginal-model class; the authors do not implement these, but nothing in the loop prevents it.","Beyond the paper, the cross-validation shortcut of holding random projections fixed across folds and updating only the data-dependent parts means reported CV results should be read as conditional on that fixed-projection choice; full per-fold updates could change the measured performance.","Beyond the paper, the package's own simulated example, which ships with the true coefficients, could power a routine diagnostic: comparing estimated and true coefficients after fitting would make the approximate-span property observable rather than assumed."],"forward_implications":["For p >> n data, no ensemble member ever fits a model on more than 2n screened variables projected to at most n/2 dimensions, so the algorithm stays computationally tractable where a direct GLM fit is infeasible or unstable.","Because coefficients are averaged in the original variable space after thresholding, the resulting object supports standard coef and predict methods and yields per-predictor effects rather than effects on uninterpretable projected features.","The S3 framework allows new screening and projection methods to be dropped in without touching the core loop; the paper shows this by adding a SIRS-based screening coefficient and a validation-selected Haar projection.","The cross-validation routine can tune both the number of ensemble models and the threshold level, including a one-standard-error rule that returns a sparser coefficient vector with nearly the same validation performance.","The internal L2 penalty on marginal models is especially useful for binomial families, where it reduces separation problems during the ensemble fit."],"supporting_citations":[{"why":"It supplies the SPAR algorithm and the data-driven random projection for GLMs that the package implements.","marker":"Parzer et al. (2024a)"},{"why":"It gives the theoretical expected-gain bound for response-informed projections and the ridge-coefficient replacement used in rp_cw(data=TRUE).","marker":"Parzer et al. (2024b)"},{"why":"It motivates screening before random projection to avoid noise accumulation and contributes the probabilistic screening idea used in the ensemble.","marker":"Mukhopadhyay and Dunson (2020)"},{"why":"It provides the sparse embedding matrix structure on which rp_cw builds.","marker":"Clarkson and Woodruff (2013)"},{"why":"It supplies sparse Rademacher random projections and the Johnson-Lindenstrauss dimension bound used for choosing the lower projection dimension.","marker":"Achlioptas (2003)"},{"why":"It establishes sure independence screening, the classical screening approach behind the fixed screening option.","marker":"Fan and Lv (2007)"},{"why":"It extends sure screening to generalized linear models, underpinning the GLM-based screening coefficients.","marker":"Fan and Song (2010)"},{"why":"It defines the random-projection ensemble classification method that the paper's extensibility example reimplements as a custom projection.","marker":"Cannings and Samworth (2017)"}],"fun_headline_variants":["spar: First R package for sparse projected GLM ensembles","spar: Scale GLM ensembles to p >> n with screening and projection","spar: Extensible high-dimensional GLM ensembles via random projections","spar: Sparse projected averaging now in R for GLM ensembles","spar: R package tames p >> n with sparse projections"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that replacing the random diagonal entries of the projection with small-penalty ridge coefficients keeps the true regression coefficients approximately recoverable after projection, a property this paper cites from its technical reports rather than proving or verifying here.","fun_headline_variants_meta":{"raw":{"variants":["spar: First R package for sparse projected GLM ensembles","spar: Scale GLM ensembles to p >> n with screening and projection","spar: Extensible high-dimensional GLM ensembles via random projections","spar: Sparse projected averaging now in R for GLM ensembles","spar: R package tames p >> n with sparse projections"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000295,"raw_usage":{"total_tokens":1647,"prompt_tokens":807,"completion_tokens":840,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":423,"completion_tokens_details":{"reasoning_tokens":747}},"tokens_in":423,"tokens_out":840,"duration_ms":7323,"temperature":1.0,"reasoning_tokens":747,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T11:51:19.091363+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run spar() on the package's included simulated data, which explicitly provides the true coefficient vector, and compare rp_cw(data = TRUE) with a data-agnostic projection such as rp_gaussian(): if the data-driven default does not produce closer coefficient recovery or better variable ranking, the approximate-span premise is unsupported. A second check is to search the package ecosystem for an existing GLM ensemble that already performs screening, random projection, and averaging, which would directly test the first-implementation claim.","supporting_citations":[{"cited_title":"Random-projection ensemble classification","cited_arxiv_id":null,"evidence_quote":"It defines the random-projection ensemble classification method that the paper's extensibility example reimplements as a custom projection."}],"review_version":1}