Sparse twoblock dimension reduction for simultaneous compression and variable selection in two blocks of variables

Sven Serneels

REVIEW 4 major objections 7 minor 17 references

Sparse twoblock dimension reduction for simultaneous compression and variable selection in two blocks of variables

T0 review · 4 major / 7 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Sparse two-block dimension reduction simultaneously compresses and selects variables in both blocks, and the paper shows it predicts multivariate responses better than dense XY-PLS and PLS2.

desk verdict A promising but unproven sparse XY-PLS variant whose load-bearing flaw is that masking loadings breaks the deflation orthogonality. read the letter →

arxiv 2411.17859 v1 pith:57EUWQYG submitted 2024-11-26 stat.ME stat.CO

Sven Serneels This is my paper · ORCID

classification stat.MEstat.CO

keywords sparsePLStwo-blockdimensionreductionvariableselectionmultivariateregressionXY-PLSsoft-thresholdingchemometricspartialleastsquares

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces a sparse version of the two-block dimension reduction method XY-PLS, which finds linear combinations of the independent variables and of the dependent variables simultaneously. By soft-thresholding the weight vectors in each block, the method performs intrinsic variable selection in both X and Y blocks. The author claims that, when uninformative variables are present, this sparse estimator estimates regression coefficients more accurately than dense XY-PLS and than multivariate PLS (PLS2), and that the advantage grows with the number of irrelevant predictors. Two chemometric applications, concrete slump and NIR biscuit dough data, are used to show improved overall predictive performance. The paper also provides an explicit algorithm, so the method can be implemented directly.

What carries the argument

The central object is Algorithm 1, the sparse twoblock PLS algorithm. It takes the dense XY-PLS weight vectors, normalizes them, applies the soft-thresholding rule of Equation (9) with sparsity parameters kappa for the response block and eta for the predictor block, and masks the corresponding loadings so that each latent direction has a single nonzero pattern. Each block is deflated separately as in XY-PLS, preserving the zero-covariance conditions of Equation (3) among scores, and the final multivariate regression coefficients are assembled from the sparse weights and loadings by Equation (8). This mechanism ties variable selection directly to the dimension-reduction step, so the same selected variables appear in weights, loadings, and coefficients.

What would settle it

A simulation with a known sparse coefficient structure in which the sparse twoblock estimator selects a large fraction of uninformative variables and yields higher prediction error than dense XY-PLS, or a direct check that the thresholded weights produce nonzero covariances between residual blocks, would refute the method's core claim.

Watch

Extended reading notes

Core claim

The paper claims that applying a soft-thresholding operation to the XY-PLS weighting vectors, with separate tunable sparsity parameters for the predictor and predictand blocks, yields an estimator that simultaneously reduces dimension and deselects uninformative variables in both blocks. The resulting sparse twoblock PLS algorithm computes sparse weights, sparse loadings, and sparse regression coefficients that remain consistent in their nonzero patterns, and the deflation steps guarantee uncorrelated scores. The author argues that this estimator outperforms dense XY-PLS and PLS2 in simulation studies, especially as the proportion of uninformative predictors increases, and that it gives better overall prediction errors on the concrete slump and NIR biscuit dough datasets. The regression coefficients are still computed by the XY-PLS formula, with sparse weights in place of the dense ones.

Load-bearing premise

The load-bearing premise is that applying soft-thresholding to the weight vectors and then masking the loadings does not break the deflation steps or the regression-coefficient formula, so the estimator still obeys the zero-covariance constraints; the paper does not prove this and calls the method 'postulated ad hoc'.

Editorial extensions

If this is right

Variable selection in both X and Y blocks becomes automatic via cross-validation of the two sparsity parameters, removing the need for manual wavelength or feature selection.
The method can be applied to settings with p > n, since the simulation shows stable coefficient estimation when the number of predictors exceeds the sample size.
Because Y-block variables can be deselected, the estimator can flag response variables that carry no joint information, which the paper notes is relevant for genomics and metabolomics.
The sparse estimator outperforms PLS2 more as uninformative predictors increase, suggesting it is particularly suited to high-dimensional data with many irrelevant variables.
The explicit algorithm allows straightforward implementation and adaptation to other PLS-family variants.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The soft-thresholding rule is one of many possible sparsity-inducing schemes; lasso-type or adaptive thresholding could yield different selection properties, but the paper does not compare them.
The claim that thresholding preserves the zero-covariance deflation structure is not proven theoretically; if a future proof fails, the advantages may be specific to the simulation design or require additional constraints.
In applications where both blocks are large and sparse, the method could serve as an exploratory tool to map which predictors relate to which responses, beyond its predictive role.
The comparison to sparse CCA in the outlook suggests the method could be recast as a sparse alternative to canonical correlation with per-block complexity control.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The new thing here is an algorithm that puts the SNIPLS soft-thresholding rule onto XY-PLS, giving sparse dimension reduction in both X and Y with separate sparsity parameters. That combination is new, and the two real-data examples are genuinely encouraging: sparse twoblock beats PLS1, PLS2, and dense XY-PLS on the slump test set and gets better or comparable R2 on the NIR biscuit data. The paper is also honest about what it does not prove: it calls the estimator "postulated ad hoc" and defers population theory to future work. Those are points in its favor.

The soft spots are real, and one of them is load-bearing. The simulation section only compares against PLS2, not dense XY-PLS, even though the abstract promises the new method outperforms its dense counterpart. The averages are over 1000 runs but there are no error bars; the sparsity parameters are fixed at 0.5 with no sensitivity analysis; no code or data are supplied. The citation pattern is fine: XY-PLS and SNIPLS are properly attributed, and the self-citations are disclosed and relevant.

The deeper problem is the masking of loadings in Algorithm 1. In dense XY-PLS, q_j is the OLS loading F_{j-1}^T u_j/(u_j^T u_j), so the deflation F_j = F_{j-1} - u_j q_j^T removes all covariance between u_j and the residual. In the sparse version, q_j is elementwise multiplied by the mask m_j. Variables excluded by the mask keep their full covariance with u_j, so the next score can remain correlated with the previous one. The same happens in the predictor block with p_i. Equations (3), (7), and the regression-coefficient formula (8) all rely on that orthogonality. This is not a missing population theorem; the algorithm as written is inconsistent with its own deflation equations. The reported gains over dense XY-PLS and PLS2 could therefore be an artifact of the masking rather than evidence for sparse two-block dimension reduction. This is fixable, but it needs to be fixed, not deferred.

Who is this for: chemometricians and practitioners who want automatic variable selection in multivariate calibration. I would send it to a serious referee, but with a clear request to address the deflation orthogonality, add dense XY-PLS to the simulations, report variability, and release code and data. As it stands, the central claim is not supported.

Referee Report

4 major / 7 minor

Summary. The paper introduces a sparse version of the XY-PLS algorithm for simultaneous dimension reduction in two blocks. It applies soft-thresholding to the weight vectors of each block, masks the loadings to the selected variables, and derives sparse regression coefficients. The method is evaluated in a simulation study and on two chemometric data sets, and is claimed to outperform dense XY-PLS and multivariate PLS2. The paper explicitly acknowledges that the estimator is postulated ad hoc and defers theoretical analysis to future work.

Significance. If the empirical claims hold, the paper offers a practical algorithm for multivariate calibration with variable selection in both X and Y blocks, and it usefully extends the less-cited XY-PLS approach. The manuscript is clearly written, the algorithm is concrete, and the two real-data examples use appropriate independent test splits. However, the simulation evidence is narrower than the abstract claims, and the masked deflation creates a technical inconsistency that needs to be resolved before the reported improvements can be attributed to sparse two-block dimension reduction.

major comments (4)

[Section 2.2, Algorithm 1, steps 1(b)vi-vii and 2(b)vi-vii; Eq. (7)] Masking q_j and p_i with m_j and n_i breaks the deflation orthogonality. In dense XY-PLS, q_j = F_{j-1}^T u_j/(u_j^T u_j) gives u_j^T F_j = 0; with q_j ⊙ m_j, u_j^T F_j = (F_{j-1}^T u_j)^T(1-m_j), which is generally nonzero. The same holds for t_i^T E_i. The sparse algorithm therefore does not satisfy Equations (3) and (4), and Equation (8) is inherited from the dense derivation without justification. Since the paper itself calls the estimator 'postulated ad hoc' and says the loadings masking is 'strictly speaking not necessary,' the authors should either prove that the masked deflation preserves the needed orthogonality, remove the masking and verify that sparse coefficients still result, or add a comparison that isolates the effect of the masking. Without this, the reported gains over dense XY-PLS may be an artifact of the implementation rather than of sparse two-block dimension reduction.
[Abstract and Section 3.3] The abstract claims that in the simulation study the new method outperforms its dense counterpart, but the simulation compares only Sparse Twoblock with PLS2; dense XY-PLS is never included in the simulation. Figures 1 and 2 and the accompanying text describe two estimators only. The central claim 'outperforms dense XY-PLS' is therefore unsupported by the simulation and should be either substantiated by adding dense XY-PLS to the simulation or removed from the abstract.
[Section 3.1, Eq. (12)] The MSEB definition appears to have a dimension mismatch. B is p×q, and the text says the metric compares coefficients for the informative dependent variables, so the slice should be \`hat B[:,1:q1]\` (or the rows should be restricted to the p1 informative predictors). As printed, the expression \`hat B[:,1:p1]\` uses p1 in the column index. Please correct the notation; this metric drives all simulation conclusions.
[Section 3.3] The simulation fixes η=κ=0.5, h=3, g=1 and reports only point averages over 1000 runs. No error bars, standard errors, or sensitivity analysis around the sparsity parameters are provided. Because the proposed method has four tuning parameters and the text draws a general conclusion that it 'increasingly outperforms PLS2,' the simulation evidence should include at least a small grid over (η,κ) and (h,g), or report variability, to show the result is not an artifact of the single chosen setting.

minor comments (7)

[Algorithm 1, step 2(b)i] The subscript in E_j is undefined in the predictor loop; it should be E_{i-1} (and similarly for step 2(b)vi's E_i).
[Section 3.1] The sentence 'Loadings are composed of ... P = [p0, ⋯, pp10⋯p2]' is garbled; please restate the construction of the true loading matrix.
[Section 3.2, Eq. (12)] The typesetting of the formula is corrupted; please use a standard norm notation such as \\| \cdot \\|_F^2.
[Section 4.2] The text reports MSE values (0.95, 1.73, 0.34) that do not appear in Table 2, which reports R2 values; use one metric consistently or give both.
[Section 4.2] The number of wavelengths selected by Cook et al. is given as both 63 and 64; please make the count consistent.
[Sections 3.3 and 4.1] Minor typographical issues: 'reported in in Cook et al.' (Section 4.1), 'compositions pf seven' (Section 4.1), and the missing word 'in which' in Section 3.3.
[General] The manuscript does not provide code or a data availability statement; for an algorithmic paper, sharing the implementation would improve reproducibility.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the sparse twoblock estimator is presented as an explicit algorithmic construction and all headline claims are assessed on out-of-sample predictions and simulations with known truth, not on quantities fitted into the estimator.

full rationale

The paper's load-bearing claims are empirical comparisons on independent test data: the Concrete Slump test set consists of 25 additional cases not used in training, and the NIR biscuit dough data use the original pre-defined 40/32 split with the usual outliers removed. Sparse twoblock hyperparameters are selected by cross-validation on the training portions only, and the reported MSE/R2 values are computed on those independent test sets. In the simulation study, regression coefficients are estimated from simulated data whose true coefficients are known by construction, so MSEB measures estimation error against a known target rather than a fitted quantity. The estimator itself is explicitly admitted to be an algorithmic construction, not a derived optimum: Section 2.2 states that 'Algorithm 1 may appear to have been postulated ad hoc' and defers population theory to future work. That candid admission is a limitation, not a circularity: no prediction in the paper is obtained by feeding the target of prediction into the algorithm's inputs. The self-citations to Serneels et al. (2004, 2024) and to the SNIPLS papers co-authored by the author are contextual: the sparsification operator is imported as an explicit building block ('analogous to how the weighting vectors are sparsified in the SNIPLS algorithm'), and no load-bearing theorem is claimed from those citations. The skeptical concern about the masking in Algorithm 1 steps vi and vii is a real methodological issue -- masking loadings after computing them means the deflation in Equation (7) no longer exactly enforces zero covariance between successive scores -- but this is a correctness or validation risk, not a circularity in the sense of a prediction reducing by construction to a fitted input. No enumerated circularity pattern applies to any specific equation or section.

Assumptions & free parameters 4 free parameters · 3 assumptions · 0 invented entities

The central claim rests on three assumptions: validity of the base XY-PLS estimator, validity of the ad hoc thresholding for the sparse version, and standard centering. The first is a literature result from Cook et al. (2023), the third is standard, and the second is unproved in this paper and explicitly deferred. Four hyperparameters (two sparsity parameters and two component counts) are tuned or fixed.

free parameters (4)

eta (sparsity for predictor block) = 0.5 in simulation; tuned by CV in examples
Controls the soft-thresholding of the X-block weight vectors in Eq. (9b). In simulations it is fixed at 0.5 without sensitivity analysis; in the real examples it is selected by cross-validation.
kappa (sparsity for response block) = 0.5 in simulation; tuned by CV in examples (0.75 for concrete, 0 for NIR)
Controls the soft-thresholding of the Y-block weight vectors in Eq. (9a). Same remarks as eta.
h (number of X components) = 3 in simulation; 5 (concrete) and 9 (NIR) by CV
Number of latent variables kept for the predictor block; chosen by cross-validation in applications and fixed to the true value in simulations.
g (number of Y components) = 1 in simulation; 3 (concrete) and 2 (NIR) by CV
Number of latent variables kept for the response block. In simulations g=1 is used even though the data are generated from three components, which is an unexplained choice.

assumptions (3)

domain assumption The XY-PLS algorithm of Cook et al. (2023) provides valid estimates of the simultaneous central subspace satisfying Equations (3).
Invoked at the beginning of Section 2.1 as the foundation on which the sparse version is built; the paper does not re-derive it.
ad hoc to paper Soft-thresholding the weight vectors and masking the loadings preserves the estimation properties of XY-PLS.
The paper states in Section 2.2 that the estimator is 'postulated ad hoc' and defers theoretical underpinnings to future work, so the preservation of zero-covariance constraints (Eq. 3) is assumed rather than shown.
standard math Data are column-centered before analysis.
Stated in Section 2.1 as a normalization assumption for the deflation steps.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Sparse twoblock dimension reduction for simultaneous compression and variable selection in two blocks of variables." pith.science (2026). https://pith.science/paper/57EUWQYG

@misc{pith2026241117859,
  author       = {Pith},
  title        = {Pith review of: Sparse twoblock dimension reduction for simultaneous compression and variable selection in two blocks of variables},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/57EUWQYG}},
  note         = {Machine review of arXiv:2411.17859}
}

read the original abstract

A method is introduced to perform simultaneous sparse dimension reduction on two blocks of variables. Beyond dimension reduction, it also yields an estimator for multivariate regression with the capability to intrinsically deselect uninformative variables in both independent and dependent blocks. An algorithm is provided that leads to a straightforward implementation of the method. The benefits of simultaneous sparse dimension reduction are shown to carry through to enhanced capability to predict a set of multivariate dependent variables jointly. Both in a simulation study and in two chemometric applications, the new method outperforms its dense counterpart, as well as multivariate partial least squares.

Figures

Figures reproduced from arXiv: 2411.17859 by the authors.

**Figure 2.** Averages over 1000 simulation runs of mean squared [PITH_FULL_IMAGE:figures/full_fig_p012_2.png] view at source ↗

**Figure 3.** Actual versus predicted values for the independen [PITH_FULL_IMAGE:figures/full_fig_p015_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

17 extracted references · 16 canonical work pages

[1]

Cook, R. D. and Forzani, L. (2021). Pls regression algorithms in the presence of nonlinearity. Chemometrics and Intelligent Laboratory Systems , 213:104307

work page 2021
[2]

D., Forzani, L., and Liu, L

Cook, R. D., Forzani, L., and Liu, L. (2023). Partial least squares for simultaneous reduction of response and predictor vectors in regression. Journal of Multivariate Analysis , 196:105163

work page 2023
[3]

D., Helland, I., and Su, Z

Cook, R. D., Helland, I., and Su, Z. (2013). Envelopes and partial least squares regression. Journal of the Royal Statistical Society: Series B (Statistical Methodology) , 75(5):851--877

work page 2013
[4]

Hoffmann, I., Filzmoser, P., Serneels, S., and Varmuza, K. (2016). Sparse and robust pls for binary classification. Journal of Chemometrics , 30(4):153--162

work page 2016
[5]

Hoffmann, I., Serneels, S., Filzmoser, P., and Croux, C. (2015). Sparse partial robust m regression. Chemometrics and Intelligent Laboratory Systems , 149:50--59

work page 2015
[6]

D., and Tsai, C.-L

Li, L., Cook, R. D., and Tsai, C.-L. (2007). Partial inverse regression. Biometrika , 94(3):615--625

work page 2007
[7]

G., Fearn, T., Miller, A

Osborne, B. G., Fearn, T., Miller, A. R., and Douglas, S. (1984). Application of near infrared reflectance spectroscopy to the compositional analysis of biscuits and biscuit doughs. Journal of the Science of Food and Agriculture , 35(1):99--105

work page 1984
[8]

Serneels, S., Croux, C., Filzmoser, P., and Van Espen, P. J. (2005a). Partial robust m-regression. Chemometrics and Intelligent Laboratory Systems , 79(1-2):55--64

work page 2005

Show all 17 references

[9]

Serneels, S., Croux, C., and Van Espen, P. J. (2004). Influence properties of partial least squares regression. Chemometrics and Intelligent Laboratory Systems , 71(1):13--20

2004
[10]

Serneels, S., Geladi, P., Moens, M., Blockhuys, F., and Van Espen, P. J. (2005b). Influence properties of trilinear partial least squares regression. Journal of Chemometrics: A Journal of the Chemometrics Society , 19(8):405--411

2005
[11]

Serneels, S., Insolia, L., and Verdonck, T. (2024). Elegant robustification of sparse partial least squares by robustness-inducing transformations. Statistics , 58(1):44--64

2024
[12]

and Van Espen, P

Serneels, S. and Van Espen, P. J. (2005). Bootstrap confidence intervals for trilinear partial least squares regression. Analytica chimica acta , 544(1-2):153--158

2005
[13]

Wegelin, J. A. (2000). A survey of partial least squares (pls) methods, with emphasis on the two-block case. University of Washington, Tech. Rep , 371

2000
[14]

and Croux, C

Wilms, I. and Croux, C. (2015). Sparse canonical correlation analysis from a predictive point of view. Biometrical Journal , 57(5):834--851

2015
[15]

and Croux, C

Wilms, I. and Croux, C. (2016). Robust sparse canonical correlation analysis. BMC systems biology , 10:1--13

2016
[16]

Wold, H. (1966). Nonlinear estimation by iterative least squares procedures. In David, F., editor, Papers in Statistics: Festschrift for J. Neyman , pages 411--444. Wiley

1966
[17]

Yeh, I.-C. (2007). Modeling slump flow of concrete using second-order regressions and artificial neural networks. Cement and Concrete Composites , 29(6):474--480

2007

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.

[1] [1]

Cook, R. D. and Forzani, L. (2021). Pls regression algorithms in the presence of nonlinearity. Chemometrics and Intelligent Laboratory Systems , 213:104307

work page 2021

[2] [2]

D., Forzani, L., and Liu, L

Cook, R. D., Forzani, L., and Liu, L. (2023). Partial least squares for simultaneous reduction of response and predictor vectors in regression. Journal of Multivariate Analysis , 196:105163

work page 2023

[3] [3]

D., Helland, I., and Su, Z

Cook, R. D., Helland, I., and Su, Z. (2013). Envelopes and partial least squares regression. Journal of the Royal Statistical Society: Series B (Statistical Methodology) , 75(5):851--877

work page 2013

[4] [4]

Hoffmann, I., Filzmoser, P., Serneels, S., and Varmuza, K. (2016). Sparse and robust pls for binary classification. Journal of Chemometrics , 30(4):153--162

work page 2016

[5] [5]

Hoffmann, I., Serneels, S., Filzmoser, P., and Croux, C. (2015). Sparse partial robust m regression. Chemometrics and Intelligent Laboratory Systems , 149:50--59

work page 2015

[6] [6]

D., and Tsai, C.-L

Li, L., Cook, R. D., and Tsai, C.-L. (2007). Partial inverse regression. Biometrika , 94(3):615--625

work page 2007

[7] [7]

G., Fearn, T., Miller, A

Osborne, B. G., Fearn, T., Miller, A. R., and Douglas, S. (1984). Application of near infrared reflectance spectroscopy to the compositional analysis of biscuits and biscuit doughs. Journal of the Science of Food and Agriculture , 35(1):99--105

work page 1984

[8] [8]

Serneels, S., Croux, C., Filzmoser, P., and Van Espen, P. J. (2005a). Partial robust m-regression. Chemometrics and Intelligent Laboratory Systems , 79(1-2):55--64

work page 2005