Pith. sign in

REVIEW 4 major objections 6 minor 16 references

Principal Component Analysis Using Structural Similarity Index for Images

T0 review · 4 major / 6 minor · reviewed 2026-08-14 · deepseek-v4-flash

Pith's one-line read Replacing Euclidean distance with SSIM in PCA yields subspaces that separate image distortions.

desk verdict The paper's central claim does not survive its own equations: the ISCA objective reduces to PCA, and the SSIM interpretation is invalid for uncentered reconstructions. read the letter →

arxiv 1908.09287 v1 pith:GCLK2GH7 submitted 2019-08-25 eess.IV cs.CVcs.LGstat.ML

classification eess.IVcs.CVcs.LGstat.ML
keywords structuralsimilarityindexprincipalcomponentanalysisimagestructuresubspacekernelmethodslearningqualityassessmentSSIMdistance
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Image quality measures like SSIM capture structural differences that mean squared error misses, but standard PCA is built on Euclidean distance and therefore does not see those structures. The paper replaces the Euclidean reconstruction error in PCA with an SSIM-based distance, block by block, and calls the result Image Structural Component Analysis (ISCA), with a kernel version for nonlinear structure. The authors claim the resulting image structure subspace separates distortion types, such as contrast stretch, Gaussian noise, luminance change, blurring, impulse noise, and JPEG artifacts, in a way that PCA and kernel PCA do not. If this holds, manifold learning gets an objective function aligned with perceived image fidelity rather than pixel error.

What carries the argument

The mechanism is the zero-mean, squared SSIM distance, written as the ratio of squared Euclidean error to the sum of squared block norms plus a constant, which equals one minus SSIM when both vectors are centered. Plugging the reconstruction into that expression gives a ratio objective that depends on the projection matrix, simplified by the orthonormality constraint. The authors minimize this objective with ADMM, using gradient descent for the subspace update and a singular-value projection to enforce orthonormality, while kernel ISCA repeats the same structure in feature space with the kernel constraint. The per-block subspaces are trained like undercomplete auto-encoders, one per block, so the projected vectors are the image structural components.

What would settle it

Compute, for any centered block and any learned orthonormal projection, both the paper's objective from Eq. (6) and the actual SSIM distance between the block and its reconstruction; whenever the reconstruction has a nonzero mean, which the orthonormality constraint does not prevent, the two numbers will differ. A concrete instance is the centered block with coordinates (1,-1,0,...,0) and a projection matrix whose columns are not parallel to the all-ones vector.

Watch

Extended reading notes

Core claim

The paper's central claim is that minimizing the SSIM distance between an image block and its reconstruction from an orthonormal subspace produces a subspace whose coordinates are meaningful measures of image structure, not just pixel energy. Concretely, ISCA solves the block-wise problem of minimizing the sum of SSIM distances between each block and its projection, subject to orthonormality, and kernel ISCA solves the same problem in a feature space after double-centering the kernel. In the reported experiments, a 1-nearest-neighbor classifier on the projected block coordinates recognizes six distortion types with higher accuracy than PCA and several kernel PCA variants, including on out-of-sample images with combined distortions. The authors state that this defines an image structure subspace which discriminates different types of image distortions.

Load-bearing premise

The load-bearing premise is that the simplified SSIM distance formula applies to a block and its reconstruction, which requires both to have zero mean; the method centers the block but never ensures the reconstruction is centered.

Editorial extensions

If this is right

  • ISCA projections give a feature space in which six common distortion types can be told apart by a 1-nearest-neighbor rule, including on images not used in training.
  • Kernel ISCA, which centers the kernel instead of the raw block, generally recognizes distortions better than linear ISCA and than RBF or sigmoid kernel PCA.
  • Reconstruction from the ISCA subspace stays close to the original image, so the subspace preserves enough structure to serve as an image model.
  • Because training is organized as one undercomplete auto-encoder per block, the same ADMM scheme extends to deeper auto-encoders with nonlinear activations, as the paper notes for future work.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A straightforward test of the method's perceptual grounding is whether ISCA coordinates predict human opinion scores on standard image quality databases; the paper does not report such a comparison.
  • If the centering gap in the objective is patched, for example by constraining the projection to have zero column sums or by using the full SSIM formula, the same block-wise ADMM machinery would still apply and might change which distortions separate.
  • The distortion-discrimination result is demonstrated on images derived from a single base image; whether the structure subspace generalizes across natural image content is an open empirical question.
  • The connection to quality assessment suggests ISCA could be used to select among denoising methods by comparing block projections, which the paper names as future work but does not test.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes Image Structural Component Analysis (ISCA) and a kernel variant (kernel ISCA) as subspace-learning methods that replace the Euclidean distance in PCA with the SSIM-based distance for images. The authors define an 'image structure subspace' intended to discriminate distortion types, derive gradient and ADMM updates for the proposed objectives, and report experiments on a dataset built from a single Lena image with six distortion types, including confusion matrices for distortion recognition and out-of-sample reconstructions. The supplementary material contains detailed algebraic derivations of the main equations.

Significance. If the central derivation were valid, the idea of replacing Euclidean distance with SSIM in subspace learning would be a conceptually interesting bridge between image quality assessment and manifold learning, and the empirical distortion-recognition results would be a useful proof of concept. The paper is clearly written in its structure, provides explicit derivations in the supplementary material, and makes its optimization formulation concrete. However, the load-bearing derivation is, in my reading, incorrect: the objective that ISCA actually minimizes is not the SSIM distance, and for a fixed block the stated objective reduces to maximizing the usual PCA projection energy. As a result, the paper's central claim that ISCA uses SSIM rather than Euclidean distance is not established, and the reported experiments do not compensate for this because they are based on one training image, in-sample evaluation, and no error bars. I therefore cannot recommend acceptance.

major comments (4)
  1. [Section 4.1, Eq. (6)] The derivation of f(U) is not valid. Equation (2) defines the SSIM distance only when both arguments have zero mean, but the pair (x, UU^Tx) is not guaranteed to satisfy this: the reconstruction UU^Tx has mean (1^T U)(U^T x)/q, and the constraint U^TU=I does not imply U^T1=0. A concrete counterexample is x=(1,-1) and u=(1,0), for which UU^Tx=(1,0) has mean 1/2. Thus Eq. (6) is not 1−SSIM(x, UU^Tx); the luminance term of Eq. (1) has been silently dropped. This invalidates the gradient in Eq. (7) and the ADMM updates in Eq. (15), which minimize an unstated objective rather than the SSIM reconstruction error claimed in Eq. (5).
  2. [Section 4.1, Eqs. (5)–(6)] Even if one accepts the algebraic expression in Eq. (6), the single-image ISCA objective is equivalent to PCA. Writing a = ||x||^2 and b = ||U^Tx||^2, Eq. (6) becomes f(U) = (a−b)/(a+b+c), which is strictly decreasing in b for fixed x and c. Minimizing f(U) is therefore equivalent to maximizing b, i.e., maximizing the projection energy of x, which is exactly the PCA objective in Eq. (3). Consequently, the paper's claim that ISCA 'uses SSIM, rather than Euclidean distance' is contradicted by its own equations; the stated formulation does not produce a subspace that differs from PCA in this single-block case.
  3. [Section 5, Eqs. (16)–(18)] The kernel version inherits the same defect. The derivation of Eq. (17) applies Eq. (2) to the pair (φ(x), Φ(X)ΘΘ^T k) without showing that the reconstruction is centered in the sense required by Eq. (2). Double-centering the kernel centers the set of dictionary points in feature space, but it does not make either φ(x) or the reconstruction Φ(X)ΘΘ^T k zero-mean over its coordinates, which is the condition under which Eq. (2) equals 1−SSIM. Moreover, the resulting objective in Eq. (17) has the same form (k_i − k_i^TΘΘ^T k_i)/(k_i + k_i^TΘΘ^T k_i + c), so minimizing it is equivalent to maximizing k_i^TΘΘ^T k_i subject to Θ^TKΘ=I, i.e., kernel PCA projection variance. The kernel ISCA claim therefore fails for the same reason as in the linear case.
  4. [Section 6, Figs. 3–4 and Table 1] The experimental evidence is too thin to support the paper's broader claims. The subspaces are trained on a single image (Lena), the confusion matrices in Fig. 3 are computed on the training images themselves, no error bars or repeated trials are reported, and Table 1 gives no comparison baseline with variability. Given the central derivation problem, these experiments cannot substantiate the claimed 'image structure subspace' that discriminates distortion types; the results would need a much more robust evaluation to be convincing even if the objective were correct.
minor comments (6)
  1. [Throughout] There are numerous typos and inconsistencies, including 'orthonomal', 'kernals', 'gradient decent', 'for for', and 'evaulate'; a careful proofread is needed.
  2. [Section 2, Eq. (2)] The notation R∋ preceding equations is nonstandard and should be replaced with a conventional statement such as 'where f(U) ∈ R is defined by...'.
  3. [Section 4.1, Eq. (15)] In the projection step, the notation uses Q_i and Ω_i for left and right singular vectors of (U_i^{(k+1)}+J_i^{(k)}), but the earlier SVD-based proximal operator in Eq. (14) uses Q and Ω in a different role; the dimensions and roles should be clarified to avoid confusion.
  4. [Fig. 2 caption] The phrase 'Lena can be seen in them by noticing scrupulously' is vague and not a quantitative statement; either remove it or replace it with a specific explanation of what the visualization shows.
  5. [Table 1] The caption defines the abbreviations O, C, G, L, B, I, J but the table entries include composite labels such as 'B + G' and 'J + L'; the definition of composite combinations should be explicit.
  6. [Section 6, paragraph beginning 'Training:'] The statement that one iteration of gradient descent suffices for Eq. (10) is presented as an empirical observation but no supporting evidence is given; this should be justified or removed.

Circularity Check

2 steps flagged · score 6.0 of 10

ISCA's SSIM objective reduces by the paper's own Eq. (6) to PCA's projection-energy objective, so the central 'structural component' method is PCA renamed.

  1. renaming known result [Section 4.1, Eq. (6); claimed contribution in Abstract]
    "R∋f(U) :=||˘x−UU⊤˘x||S = ˘x⊤(I−UU⊤) ˘x / (˘x⊤(I +UU⊤) ˘x +c). ... 'We propose Image Structural Component Analysis (ISCA) and also kernel ISCA by using SSIM, rather than Euclidean distance, in the formulation of PCA.'"

    For a fixed centered block x, let a=||x||^2 and b=||U^T x||^2. Eq. (6) is exactly f(U)=(a−b)/(a+b+c), whose derivative w.r.t. b is −(2a+c)/(a+b+c)^2<0. Hence minimizing f(U) over orthonormal U is equivalent to maximizing b, the squared length of the PCA projection. The resulting stationarity condition is xx^T U=UΛ, the PCA eigenproblem. Thus the 'SSIM rather than Euclidean distance' formulation is a monotone transform of PCA's Euclidean objective; the claimed image-structure subspace is PCA by construction, not a new SSIM-based method.

  2. renaming known result [Section 5, Eq. (17)]
    "R∋f(Θi) :=||φ(˘xi)−Φ( ˘Xi)ΘiΘ⊤ i ki||S = ki−k⊤ i ΘiΘ⊤ i ki / (ki +k⊤ i ΘiΘ⊤ i ki +c)."

    After the paper's kernel normalization, ki=1, so Eq. (17) reads f=(1−d^TΘΘ^T d)/(1+d^TΘΘ^T d+c) with d=ki; this is again strictly decreasing in the squared kernel-projection norm d^TΘΘ^T d. Minimizing it is therefore equivalent to maximizing the kernel PCA projection energy. Kernel ISCA inherits the same reduction: the SSIM distance in the objective is a monotone stand-in for the Euclidean (kernel) projection norm, so the kernel variant is kernel PCA under a monotone transform.

full rationale

The paper's derivation chain is internally explicit and relies on no fitted parameter that is later renamed as a prediction; its citations to prior SSIM work (Otero, Brunet, Wang, Vrscay) are not self-citations of the present authors, so no self-citation chain carries the argument. The genuine problem is algebraic: substituting the zero-mean SSIM distance of Eq. (2) into the reconstruction objective yields, by Eq. (6), f=(a−b)/(a+b+c), a strictly decreasing function of the PCA projection energy b. Hence the advertised 'SSIM rather than Euclidean distance' formulation has exactly the same optima and fixed points as PCA (and kernel ISCA reduces to kernel PCA). This is a renaming of a known result rather than a new structural subspace method. A separate, non-circular correctness flaw is that Eq. (2) requires both arguments to be centered, but UU^T x is not centered under U^TU=I, so f(U) is not actually 1−SSIM(x,UU^T x) as claimed. The in-sample confusion-matrix evaluation is also weak evidence, but it is not circular because the 1NN labels are not used in training.

Assumptions & free parameters 5 free parameters · 4 assumptions · 1 invented entities

The method's hyperparameters (q, p, rho, eta, epochs) are chosen by hand and are not derived or cross-validated. The main unstated assumption is that the reconstruction UU^Tx is centered so that the SSIM distance simplification applies. The kernel version relies on standard kernel trick assumptions. The 'image structure subspace' is a new conceptual entity with no independent falsifiable handle.

free parameters (5)
  • block size q = 64
    8x8 blocks chosen by analogy to refs [6,11]; no sensitivity analysis is given.
  • subspace dimension p = 4
    Rank of the block projection; chosen by hand with no model selection.
  • ADMM penalty rho = 1 for ISCA, 0.1 for kernel ISCA
    Hand-chosen; no tuning study is reported.
  • learning rate eta = 0.1
    Hand-chosen for the single gradient step inside ADMM; no convergence analysis.
  • number of epochs and termination epsilon = not specified
    The paper gives a termination criterion but does not report the threshold or the number of epochs used in the experiments.
assumptions (4)
  • domain assumption SSIM distance formula in Eq. (2) requires centered vectors.
    Eq. (2) is taken from refs [6,5,7]; ISCA uses it for x and UU^Tx but does not enforce centered reconstruction, so the objective is only approximately SSIM.
  • domain assumption ADMM with one gradient descent step converges to a solution of the nonconvex ISCA problem.
    The paper asserts that one gradient iteration suffices based on experiments, but provides no convergence proof for the nonconvex objective.
  • standard math Projection directions in feature space can be expressed as linear combinations of mapped training points.
    Standard kernel representation theorem, invoked in Section 5 via ref [16].
  • ad hoc to paper Training on one image, Lena, yields subspaces representative of other natural images.
    The entire training set is derived from one source image; generalization to other images is assumed but untested.
invented entities (1)
  • Image structure subspace
    purpose: A subspace whose projection coordinates are claimed to capture structural distortions and discriminate distortion types.
    The concept is defined by the proposed optimization, not by any external measurement; its meaningfulness is supported only by single-image experiments in this paper.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Principal Component Analysis Using Structural Similarity Index for Images." pith.science (2026). https://pith.science/paper/GCLK2GH7

@misc{pith2026190809287,
  author       = {Pith},
  title        = {Pith review of: Principal Component Analysis Using Structural Similarity Index for Images},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/GCLK2GH7}},
  note         = {Machine review of arXiv:1908.09287}
}
read the original abstract

Despite the advances of deep learning in specific tasks using images, the principled assessment of image fidelity and similarity is still a critical ability to develop. As it has been shown that Mean Squared Error (MSE) is insufficient for this task, other measures have been developed with one of the most effective being Structural Similarity Index (SSIM). Such measures can be used for subspace learning but existing methods in machine learning, such as Principal Component Analysis (PCA), are based on Euclidean distance or MSE and thus cannot properly capture the structural features of images. In this paper, we define an image structure subspace which discriminates different types of image distortions. We propose Image Structural Component Analysis (ISCA) and also kernel ISCA by using SSIM, rather than Euclidean distance, in the formulation of PCA. This paper provides a bridge between image quality assessment and manifold learning opening a broad new area for future research.

Figures

Figures reproduced from arXiv: 1908.09287 by the authors.

Figure 1
Figure 1. Examples from the training dataset: (a) original image, (b) contrast stretched, (c) Gaussian noise, (d) luminance enhanced, (e) Gaussian blurring, (f) salt & pepper impulse noise, and (g) JPEG distortion [PITH_FULL_IMAGE:figures/full_fig_p009_1.png] view at source ↗
Figure 2
Figure 2. The first dimension of the trained (a) U, (b) V , and (c) J for ISCA. 6 Experiments Training Dataset: We formed a dataset out of the standard Lena image. Six different types of distortions were applied on the original Lena image (see [PITH_FULL_IMAGE:figures/full_fig_p009_2.png] view at source ↗
Figure 3
Figure 3. Confusion matrices for recognition of distortion types with a 1NN classifier used in the subspace. Matrices (a) and (e) correspond to ISCA and PCA (or linear￾kernel PCA), respectively. Matrices (b) to (d) are for kernel ISCA with linear, RBF, and sigmoid kernels. Matrices (f) and (g) are for kernel PCA with RBF and sigmoid kernels. The 0 label in matrices correspond to the original image and the labels 1 to 6 are th… view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Out-of-sample images with different types of distortions having MSE = 500: (1) stretching contrast, (2) Gaussian noise, (3) luminance enhancement, (4) Gaussian blurring, (5) impulse noise, (6) JPEG distortion, (7) Gaussian blurring + Gaussian noise, (8) Gaussian blurri…
Figure 5
Figure 5. Figure 5: Reconstruction of images in ISCA. Reconstruction of the training images (a), (b), and (c) are shown in (d), (e), and (f), respectively. The reconstruction of out-of￾sample images shown in [PITH_FULL_IMAGE:figures/full_fig_p011_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

16 extracted references · 6 canonical work pages

  1. [1]

    IEEE signal processing magazine 26(1) (2009) 98–117

    Wang, Z., Bovik, A.C.: Mean squared error: Love it or leave it? a new look at signal fidelity measures. IEEE signal processing magazine 26(1) (2009) 98–117

  2. [2]

    IEEE transactions on image processing 13(4) (2004) 600–612

    Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing 13(4) (2004) 600–612

  3. [3]

    Synthesis Lectures on Image, Video, and Multimedia Processing 2(1) (2006) 1–156

    Wang, Z., Bovik, A.C.: Modern image quality assessment. Synthesis Lectures on Image, Video, and Multimedia Processing 2(1) (2006) 1–156

  4. [4]

    In: Handbook of Convex Optimization Methods in Imaging Science

    Brunet, D., Channappayya, S.S., Wang, Z., Vrscay, E.R., Bovik, A.C.: Optimizing image quality. In: Handbook of Convex Optimization Methods in Imaging Science. Springer (2018) 15–41

  5. [5]

    IEEE Transactions on Image Processing 21(4) (2012) 1488– 1499

    Brunet, D., Vrscay, E.R., Wang, Z.: On the mathematical properties of the struc- tural similarity index. IEEE Transactions on Image Processing 21(4) (2012) 1488– 1499

  6. [6]

    In: International Conference Image Analysis & Recognition, Springer (2014) 167– 176

    Otero, D., Vrscay, E.R.: Unconstrained structural similarity-based optimization. In: International Conference Image Analysis & Recognition, Springer (2014) 167– 176

  7. [7]

    In: International Conference Image Analysis and Recognition, Springer (2011) 100–110

    Brunet, D., Vrscay, E.R., Wang, Z.: A class of image metrics based on the struc- tural similarity quality index. In: International Conference Image Analysis and Recognition, Springer (2011) 100–110

  8. [8]

    Springer, New York (2002)

    Jolliffe, I.: Principal component analysis. Springer, New York (2002)

Show all 16 references
  1. [9]

    Cambridge university press (2004)

    Boyd, S., Vandenberghe, L.: Convex optimization. Cambridge university press (2004)

  2. [10]

    Foundations and Trends R© in Machine learning 3(1) (2011) 1–122

    Boyd, S., Parikh, N., Chu, E., Peleato, B., Eckstein, J., et al.: Distributed opti- mization and statistical learning via the alternating direction method of multipliers. Foundations and Trends R© in Machine learning 3(1) (2011) 1–122

  3. [11]

    In: International Conference Image Analysis & Recognition, Springer (2018) 20–29

    Otero, D., La Torre, D., Michailovich, O.V., Vrscay, E.R.: Alternate direction method of multipliers for unconstrained structural similarity-based optimization. In: International Conference Image Analysis & Recognition, Springer (2018) 20–29

  4. [12]

    Foundations and Trends R© in Opti- mization 1(3) (2014) 127–239

    Parikh, N., Boyd, S.: Proximal algorithms. Foundations and Trends R© in Opti- mization 1(3) (2014) 127–239

  5. [13]

    Volume 1

    Goodfellow, I., Bengio, Y., Courville, A.: Deep learning. Volume 1. MIT press Cambridge (2016)

  6. [14]

    The annals of statistics (2008) 1171–1220

    Hofmann, T., Sch¨ olkopf, B., Smola, A.J.: Kernel methods in machine learning. The annals of statistics (2008) 1171–1220

  7. [15]

    In: Pacific-Asia Conference on Knowledge Discovery and Data Mining, Springer (2010) 362–373

    Ah-Pine, J.: Normalized kernels as similarity indices. In: Pacific-Asia Conference on Knowledge Discovery and Data Mining, Springer (2010) 362–373

  8. [16]

    Principal Component Analysis Using Structural Similarity Index for Images

    Alperin, J.L.: Local representation theory: Modular representations as an intro- duction to the local representation theory of finite groups. Volume 11. Cambridge University Press (1993) 8 Supplementary Material This section is the supplementary material for the paper “Principa...

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.