{"id":"206d9350-ed03-414d-ae6e-e977616ca982","arxiv_id":"2509.07300","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Ranked-sparsity and hybrid LASSO principal components regression improve fMRI task classification over standard LASSO PCR in several tasks, with gains in cross-validated deviance of up to 51.7%.","lead":"This paper proposes LASSO variants that weight brain-scan components by their variance, plus a hybrid that also uses raw voxels. On three fMRI datasets the new models often beat the standard approach, but the evaluation method may inflate the gains.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Reported gains may be tuning-selection artifacts: §2.5 never separates tuning from evaluation, so the 51.7%/7.3% figures could reflect the minimum of the CV criterion rather than true out-of-sample performance.","rationale":"The reader identified the same load-bearing issue in §2.5, and my independent reading supports it: the model-selection and evaluation protocol as written does not guarantee unbiased out-of-sample estimates. The mathematical construction (Eqs. 1–4) is coherent, and the information-parity derivation is a plausible contribution, so I do not see an internal inconsistency that would warrant rejection. But because the headline claims are empirical performance claims, the ambiguous nested-CV description is the key risk. The paper itself lists no independent holdout and possible PCA leakage in the Discussion, making this a self-acknowledged limitation. A nested-CV reanalysis (or code release demonstrating the intended nested loop) would settle it. Thus the verdict remains conditional pending that check.","tokens_in":14193,"tokens_out":7326,"duration_ms":91284,"concrete_test":"Implement a strictly nested CV: for each outer fold, run the full tuning procedure (γ, τ, λ inner CV) on the training fold only—ideally also recomputing PCA inside the training fold—then fit the selected model and evaluate once on the untouched outer test fold. Recompute Figure 2 / S3 metrics. If JSRL's 51.7% deviance-R² and 7.3% AUC gains shrink to ≤10% / ≤1% or reverse, the central claim is not supported. Releasing code and data would make this check reproducible.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"Section 2.5 (Model Comparison) states that for SRL, γ is chosen by cross-validated deviance over 27 candidate values, with λ selected by 10-fold CV for each γ; for JSRL, γ is chosen as the value with minimum CV deviance, τ by a 'final cross-validation step,' and 'the final model selected for prediction used the parameter values ... from the model which minimized the cross-validated deviance in this step.' Nowhere is an outer loop described that keeps the test folds of the reported performance untouched during this selection. If the reported AUC/deviance is the same quantity minimized during tuning, the headline 'up to 51.7% improvement in CVD R² and 7.3% in AUC' is the minimum over many noisy CV estimates, which is expected to be optimistically biased even under the null, particularly with n=68–82 and many candidate configurations. Because SRL/JSRL have many more tuning configurations than PCL (γ and τ grids vs. only λ), this bias is asymmetric and favors the proposed methods. The Discussion's concession that evaluation is only through CV without an independent holdout, and that PCA may leak, reinforces this concern. Since the central claim is purely empirical, this protocol ambiguity is load-bearing.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes two modifications to LASSO principal components regression for fMRI decoding: Sparsity-Ranked LASSO (SRL), which assigns index-based penalty weights to principal components, and Joint Sparsity-Ranked LASSO (JSRL), which adds voxel-level predictors with voxel penalties calibrated by an 'information parity' formula. The methods are compared against principal-component LASSO, voxel LASSO, and voxel elastic net on 9 binary classification tasks from three fMRI studies. The authors report that JSRL improves cross-validated deviance R² by up to 51.7% and AUC by up to 7.3% relative to PCL, and that sparsity-ranked models often match or outperform the alternatives. The paper includes qualitative brain-activation maps and region annotations for emotion-regulation tasks.","tokens_in":14476,"tokens_out":2907,"duration_ms":37406,"significance":"If the reported gains are genuine, the proposed methods offer a simple, drop-in modification to a widely used MVPA tool, with potential practical value in decoding mental states from fMRI. The ranking idea is intuitive and the information-parity formula provides a principled starting point for the voxel penalty. However, the central claim is purely empirical, and the evidence is compromised by an ambiguous evaluation protocol: hyperparameters are selected by minimizing cross-validated deviance, and the same deviance is then reported as the performance metric. With no clearly separated outer loop or independent holdout, the headline improvements may be optimistically biased. The paper does not currently provide code or data, further limiting verification.","major_comments":[{"comment":"The evaluation protocol does not separate hyperparameter tuning from performance estimation. For SRL, γ is chosen by minimizing cross-validated deviance; for JSRL, γ and τ are likewise selected by cross-validated deviance, and the 'final model selected for prediction' is the one minimizing that same criterion. No outer loop is described that keeps evaluation folds untouched during tuning. The reported AUC and deviance thus appear to be the minimum over many candidate configurations, which is expected to be optimistically biased, especially with n=68–82 and grids of 27 γ values and multiple τ values. The Discussion's admission that evaluation is only through CV without an independent holdout reinforces this concern. The authors should either describe a fully nested CV scheme in which tuning occurs inside each training fold, or evaluate on an independent holdout, and report the correspondi","section":"§2.5 (Model Comparison)"},{"comment":"The 'information parity' derivation is not load-bearing for the empirical claim because the resulting r_IP is only a starting point; the final r = τ·r_IP is tuned by cross-validation. This is acceptable, but the text should be clearer that Eq. (4) is a heuristic initialization, not a guarantee of optimal performance. As written, calling it 'information parity' may overstate its role. The derivation itself appears algebraically correct, but its connection to predictive accuracy is unestablished.","section":"§2.2.1, Eq. (4)"},{"comment":"The authors acknowledge that 'model performance was evaluated solely through cross-validation, without validation in an independent holdout dataset' and that PCA 'introduces a potential avenue for data leakage.' These admissions directly bear on the validity of the abstract's quantitative claims. Since the abstract presents the 51.7% and 7.3% improvements as the main result, the manuscript should either provide the nested-CV or holdout evidence needed to substantiate those numbers, or substantially temper the claims.","section":"Discussion, limitations paragraph"}],"minor_comments":[{"comment":"The text says 'A total of five modeling methods were applied' but then 'The construction of these six models'—the count is inconsistent. Also, 'left panel of Figure 1' appears after Figure 1 has already been referenced; check cross-references.","section":"§2.5, Figure 1 caption"},{"comment":"Typo: 'voxel indeces' should be 'voxel indices.' Minor.","section":"§2.2.1"},{"comment":"The description of the repeated 10-run procedure is vague: 'each run using a different random seed to determine training/testing splits during 10-fold cross-validation.' It is unclear whether tuning is performed separately within each training split or once on the full data. This ambiguity is part of the major comment, but clarifying the exact loop structure would help.","section":"§2.5"},{"comment":"The Table 1 caption calls τ a 'multiplier' but the text calls it 'rescaling factor.' Consistent terminology would improve readability. Also, reporting only averages over seeds loses information about variability across seeds; consider including SDs or ranges.","section":"§3, Table 1"},{"comment":"Code availability is deferred to acceptance. For a methods paper, sharing code at submission is important for reproducibility; consider making it available in a repository or as supplementary material.","section":"Data and Code Availability"}],"recommendation":"major_revision","confidential_remarks":"The central issue is the evaluation protocol. If the authors can demonstrate a properly nested CV or provide an independent holdout, the paper may be publishable. If not, the headline improvements are likely selection artifacts. I also note that the 'information parity' framework is a minor contribution, not a theoretical justification for performance; the paper should be reframed accordingly. The fit to a statistical ML journal is reasonable given the methodological emphasis, but the missing code release and ambiguous CV protocol are serious concerns."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's the short version: the ranked-sparsity idea is a sensible extension of LASSO PCR and the JSRL hybrid is genuinely interesting, but the empirical payoff is not yet credible because the evaluation protocol as written doesn't separate hyperparameter tuning from performance reporting. The 51.7% and 7.3% headline numbers could be the minimum of a noisy CV criterion rather than true out-of-sample gains.\n\nWhat's new: the W_k = k^gamma weighting on PC indices is a clean, interpretable way to encode the prior that earlier components carry more signal, and the information-parity derivation for r_IP is a nice touch. The JSRL model that combines PCs and raw voxels under one penalty is a plausible way to recover localized signal that PCs smear out. The paper is also transparent about its limitations—no independent holdout, possible PCA leakage, qualitative interpretability.\n\nWhere the soft spots are: the stress-test note is on target. Section 2.5 says gamma is chosen by cross-validated deviance over 27 values, lambda by 10-fold CV, and tau by a 'final cross-validation step,' with the final model being the one that minimized CV deviance. There's no sign of an outer loop that leaves the evaluation folds untouched. With n=68–82 and a large configuration grid, that's a recipe for optimistic bias, and since SRL/JSRL have many more configurations than PCL, the bias would favor them. The paper's own discussion admits the evaluation is only CV and PCA may leak. So the central empirical claim is conditional at best.\n\nThat said, this is fixable. The method is not broken; the protocol as described is ambiguous. If the authors resubmit with a properly nested CV, or an independent test set, and the gains persist, this would be a useful drop-in for MVPA. Code and data availability also need to come with it.\n\nBottom line: worth sending to reviewers, but they should be told to focus on the CV design. I'd give it a maybe for reading group—the method is worth chewing on, but the evaluation will dominate the conversation.","headline":"A sensible ranked-sparsity PCR extension with a genuinely interesting JSRL hybrid, but the headline gains rest on a CV protocol that may be circular; worth refereeing with a demand for nested CV and code.","tokens_in":14974,"tokens_out":2257,"would_cite":false,"duration_ms":25926,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["62J07"],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that giving principal components index-dependent LASSO penalties, rather than a uniform penalty, improves fMRI-based decoding of mental state, with reported gains up to 51.7% in cross-validated deviance R2.","keywords":["fMRI","multi-voxel pattern analysis","LASSO PCR","principal components regression","sparsity ranking","penalized logistic regression","cross-validation","brain decoding"],"falsifier":"Run a strictly nested cross-validation on the same three datasets where the outer test folds are never touched while selecting gamma, r, tau, and lambda, and compare SRL and JSRL against PCL on the untouched folds; if the improvement in deviance R2 drops to near zero or reverses, the central claim fails.","tokens_in":14101,"feed_emoji":"🧠","tokens_out":6941,"duration_ms":77563,"temperature":0.7,"pith_summary":"The paper tries to show that standard LASSO principal-components regression used in fMRI multi-voxel pattern analysis fails when task-relevant signal is concentrated in a few components, because it penalizes all components equally. It proposes sparsity-ranked LASSO PCR (SRL), which makes the L1 penalty a function of the component index, and Joint Sparsity-Ranked LASSO (JSRL), which adds individual voxels as predictors alongside components with a voxel penalty set by an information-parity calculation. Across three fMRI tasks, the authors report that SRL and JSRL match or beat standard LASSO PCR on all nine binary classification problems, with JSRL improving cross-validated deviance R2 by up to 51.7% and AUC by up to 7.3%. If these gains hold out of sample, the method offers a drop-in replacement that is both more accurate and more interpretable, localizing signal to core voxel clusters.","feed_headline":"Ranked-sparsity LASSO boosts fMRI decoding by up to 51.7%","feed_subtitle":"A component-indexed penalty and a voxel-plus-PC hybrid beat uniform LASSO PCR on emotion, reward, and risk tasks.","key_machinery":"The central object is the ranked penalty function W_k (k is the principal-component index, gamma shapes the penalty), together with the information-parity rescaling r_IP = -log(Σ_{k=1}^K k^{-2|gamma|}) / (2 log V) + 1/2 that balances prior Fisher information between the PC and voxel modalities in JSRL. The penalty function formalizes the implicit practice of preferring high-variance components; the parity formula gives a principled starting point for tuning the relative weight of voxels, with tau used as a searchable multiplier.","core_discovery":"Building on the observation that standard LASSO PCR assigns equal prior weight to every principal component, the paper defines a penalty weight W_k = k^gamma for gamma >= 0 (and its mirror for gamma < 0), so the model can express a prior that early components are more or less likely to carry task signal. JSRL extends this by concatenating PCs and raw voxels, assigning voxels a constant penalty scaled by V^r, and choosing the rescaling r so that the PC and voxel modalities contribute equal prior Fisher information; a tunable multiplier tau then relaxes or tightens that parity. The paper reports that on emotion regulation, monetary incentive, and risky-gain tasks, SRL and JSRL consistently mat","pith_inferences":["If ranked sparsity generalizes, the same index-weighted penalty could be applied to other dimension-reduction methods, such as PLS or sparse PCA, and to non-neuroimaging high-dimensional data where variance-ordered components coexist with raw features.","The information-parity formula suggests a principled way to set group penalties between any two feature modalities, not just PCs versus voxels; one could test whether parity-based initialization improves tuning stability in other multimodal settings.","The reported tau values below one across datasets imply that, in practice, voxels should receive weaker penalties than parity suggests; a testable extension is whether tau tends to correlate with the true spatial sparsity of the signal.","A fully independent replication with an external holdout dataset would clarify whether the 51.7% deviance-R2 gain is real, a step the authors flag as future work."],"forward_implications":["SRL and JSRL can be dropped into existing LASSO-PCR pipelines with only a change in penalty weights, so fMRI labs can adopt the method without new preprocessing.","The selected gamma for a task is interpretable: near-zero gamma means no benefit from ranking, while gamma > 0 indicates the task signal is concentrated in early components; future studies can use it to characterize tasks.","JSRL's voxel-level weights produce more localized, centered predictive maps, suggesting it may reduce edge and partial-volume noise automatically.","Because JSRL matched or outperformed PCL in all nine tasks, a researcher who currently uses uniform LASSO PCR has no accuracy reason to avoid ranked sparsity."],"supporting_citations":[{"why":"Supplies the ranked-sparsity regularization framework that SRL adapts from grouped covariates to component indices.","marker":"Peterson & Cavanaugh (2022)"},{"why":"Defines the LASSO baseline whose uniform penalty SRL and JSRL modify.","marker":"Tibshirani (1996)"},{"why":"Provides the glmnet implementation used to fit all penalized logistic regression models.","marker":"Friedman et al. (2010)"},{"why":"Established the MVPA and LASSO-PCR style decoding of mental state that this paper aims to improve.","marker":"Wager et al. (2013)"},{"why":"Contributes the Emotion Regulation fMRI dataset and contrast maps used for evaluation.","marker":"Rieck et al. (2024)"},{"why":"Contributes the Monetary Incentive Delay fMRI dataset used for evaluation.","marker":"Kirk-Provencher et al. (2024)"},{"why":"Contributes the Risky Gains fMRI dataset and links insula activation to relapse, motivating the decoding task.","marker":"Gowin et al. (2014)"}],"fun_headline_variants":["Ranked LASSO components boost fMRI decoding up to 51.7%","Hybrid JSRL method improves brain activation decoding","Sparsity-ranked PCR sharpens fMRI predictions","Component-indexed penalty lifts LASSO performance in fMRI","New JSRL model enhances mental-state decoding from scans"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The claim that SRL and JSRL outperform PCL rests on the assumption that the reported cross-validated AUC and deviance are unbiased out-of-sample estimates; because the paper tunes gamma, r, tau, and lambda without a clearly separated outer evaluation loop and uses no independent holdout, leakage could inflate the gains.","fun_headline_variants_meta":{"raw":{"variants":["Ranked LASSO components boost fMRI decoding up to 51.7%","Hybrid JSRL method improves brain activation decoding","Sparsity-ranked PCR sharpens fMRI predictions","Component-indexed penalty lifts LASSO performance in fMRI","New JSRL model enhances mental-state decoding from scans"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000627,"raw_usage":{"total_tokens":2787,"prompt_tokens":844,"completion_tokens":1943,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":588,"completion_tokens_details":{"reasoning_tokens":1863}},"tokens_in":588,"tokens_out":1943,"duration_ms":17629,"temperature":1.0,"reasoning_tokens":1863,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-04T22:26:29.353091+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run a strictly nested cross-validation on the same three datasets where the outer test folds are never touched while selecting gamma, r, tau, and lambda, and compare SRL and JSRL against PCL on the untouched folds; if the improvement in deviance R2 drops to near zero or reverses, the central claim fails.","supporting_citations":[],"review_version":1}