Pith. sign in

REVIEW 4 major objections 6 minor 1 cited by

A Fast Iterative Robust Principal Component Analysis Method

T0 review · 4 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper introduces FIR, a fast iterative robust location and covariance estimator, and constructs FIR-PCA from it, claiming dependable principal component analysis even when a large fraction of data points are outliers.

desk verdict Genuinely new robust-PCA algorithm with credible point-outlier results, but a load-bearing well-definedness gap and an incomplete equivariance proof need fixing before publication. read the letter →

arxiv 2506.16013 v1 pith:TNAXRRF6 submitted 2025-06-19 cs.CE math.STstat.TH

Timbwaoga Aime Judicael Ouermi , Jixian Li , Chris R. Johnson This is my paper · ORCID
classification cs.CEmath.STstat.TH MSC 62H2562G35
keywords robustPCAincrementalprojectiondepthoutlierdetectionlocationestimationcovariancedimensionalityreductionorthogonalequivariance
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper introduces FIR, a location-and-covariance estimator built for outlier-contaminated data, and constructs a robust PCA (FIR-PCA) from it. The estimator starts with the m points having the largest projection depth, then grows an inlier set one batch at a time: each iteration projects the remaining points into an incremental PCA space and admits the m points closest to the current inlier subset under a singular-value-scaled distance and within a box tied to the two leading principal directions. The authors claim that this selection keeps the estimated mean and covariance accurate when 40% of the data are point outliers, a regime where the DetMCD and depth-based FDB alternatives degrade, while runtime stays close to the cheaper depth-based methods. If the claim holds, FIR-PCA gives a practical way to do dimensionality reduction and outlier diagnostics on datasets that classical PCA would distort, without paying the concentration-step cost of minimum-covariance-determinant approaches.

What carries the argument

The load-bearing mechanism is the iteration in Algorithm 1 that turns a depth-based seed into an inlier set. Projection depth (Eq. 1) supplies affine equivariant starting indices; incremental PCA (Ross et al. 2008) supplies batch updates of the subspace and singular values; the scaled distance $d_i^{(k)}$ (Eq. 2) and the 0.5-expanded selection box (Eq. 3) determine which points are accepted next. The box, computed from the two leading singular vectors of the current inlier subset, is the device that keeps the search close to the data's dominant structure; Theorem 1's induction uses the invariance of the IPCA projections and singular values under rotation and translation to prove that the whole selection chain, and therefore the FIR estimates, is orthogonal equivariant and permutation invariant.

What would settle it

Take a dataset whose true inliers lie in a narrow elongated band, place 40% point outliers just outside the 0.5-expanded box, run Algorithm 1, and check whether every iteration finds m unselected points inside the box; if any iteration selects fewer than m points or pulls in an outlier because the box is empty, the claim that FIR always builds a reliable inlier set is false.

Watch

Extended reading notes

Core claim

On its own terms, the paper is establishing that a greedy, depth-seeded, incrementally updated inlier subset is enough to make PCA reliable under heavy point-outlier contamination. The procedure is Algorithm 1: initialize with the highest projection-depth points, fit an IPCA subspace to the selected set, project the unselected points, rank them by the scaled distance $d_i^{(k)} = \sum_j (\hat{Z}_{i,j}^{(k)}/s_j^{(k)})^2$, and repeatedly accept the m points with smallest distance that fall inside the selection box defined by the two leading IPCA coordinates expanded by a factor of 0.5. After $h = \alpha n$ points are collected, the mean and covariance of that subset are used to form robust PCA scores, variances, and outlier distances. The central claim is that the resulting FIR and FIR-PCA are orthogonal equivariant and permutation invariant (Theorem 1), and that at high outlier fractions they preserve the underlying structure better than DetMCD-PCA and FDB-PCA, with typical runtime between the two.

Load-bearing premise

The algorithm assumes that at every iteration at least m unselected points fall inside the selection box defined in Equation 3; if the box ever contains fewer than m points, the next step is undefined, and the paper does not guarantee this cannot happen.

Editorial extensions

If this is right

  • For datasets with 40% point outliers, the paper reports that FIR's location error and KL divergence stay close to the clean-data values, while DetMCD and FDB deteriorate (Tables 1-2, Figures 3-4).
  • The stated complexity of Algorithm 1 is $O((\kappa+\tau)np+np^2)$, which avoids the $O(\Gamma(p^3+np^2))$ cost of MCD concentration steps; measured runtimes show FIR slightly slower than FDB and far faster than DetMCD (Figure 6).
  • FIR is orthogonal equivariant and permutation invariant (Theorem 1), so rotating, translating, or shuffling the data does not change the selected inlier set or the resulting estimates.
  • On the octane, forged-bank-note, TopGear, and Philips datasets, FIR-PCA flags the known outliers with runtimes of roughly 5-44 ms, while DetMCD-PCA takes 56-692 ms in these examples.
  • The batch size $m$ controls the speed-accuracy trade-off: values near 10% of the data give most of the speed benefit without accuracy loss, and values near $p$ are the conservative choice when speed is less important.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The 0.5 expansion factor is a design parameter the paper keeps fixed; a natural extension is to test whether smaller expansion rejects borderline outliers more aggressively and larger expansion prevents the selection box from emptying at high contamination, since Algorithm 1 has no fallback if the box holds fewer than m points.
  • Because the projection-depth computation samples directions from the unit sphere, FIR carries a stochastic component that the theoretical analysis treats as fixed; a stability check across random-direction seeds would quantify how reproducible the selected inlier set H is.
  • The selection box uses only the two leading IPCA singular vectors, so the method's guarantees are most natural when the underlying structure is approximately two-dimensional; a testable variant would define the box in the leading r directions and compare robustness as r grows.
  • The paper states FIR is not designed for partial (cellwise) outliers, so a fair inference is that its target regime is rowwise contamination; handling per-feature corruption would likely need a separate feature-scaling step before the iterative selection.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes FIR, a fast iterative robust location and covariance estimator, and FIR-PCA, a robust PCA built on it. FIR selects an initial subset of m points with highest projection depth, then iteratively adds the m unselected points with smallest scaled distance in an incremental PCA (IPCA) space, restricted to a selection box formed by expanding the bounding box of the current selected points by a factor of 0.5 along the first two IPCA axes. The final subset is used to estimate location and covariance, from which robust PCA scores and distances are derived. The authors state that FIR is orthogonal equivariant and permutation invariant, analyze its computational complexity as O((κ+τ)np + np^2), and compare it against DetMCD and FDB on simulated and real datasets, reporting better robustness for point outliers at 40% contamination.

Significance. If the algorithmic gaps are closed, the paper would make a useful practical contribution: FIR is much faster than DetMCD, and the simulations in Tables 1–2 and Figures 3–5 show that it is more robust than DetMCD and FDB specifically under point outliers at 40% contamination, while remaining comparable for cluster and radial outliers. The paper is honest about limitations (p > n, partial outliers) and provides an explicit complexity analysis. The main missing elements are a rigorous well-definedness argument for the selection step and a complete equivariance proof; until those are supplied, the theoretical claims are conditional and the empirical claims rest on an incompletely specified algorithm.

major comments (4)
  1. [Algorithm 1, Step 3(d), Eq. (3)] Algorithm 1 Step 3(d) selects the m unselected points with smallest scaled distance that lie in the selection box defined by Eq. (3), but nothing guarantees that at least m unselected points fall inside this box. The box is only a 50% expansion of the bounding box of the already-selected points on the first two IPCA axes; for discrete data, tightly clustered initial points, or distant outliers, the candidate set can have size less than m. In that case Step 3(d) has no valid output, and neither the stopping rule nor any fallback (e.g., enlarging the box, taking nearest points outside the box, or terminating early) is specified. Because both FIR location/covariance and FIR-PCA call Algorithm 1, this well-definedness gap is load-bearing for the central claims of the paper. Please add a fallback or a condition under which the candidate set always has size at least m, and prove that condition.
  2. [Section 3.1, Theorem 1 proof, Eqs. (14)-(17)] The proof of orthogonal equivariance is incomplete. Equations (14)-(15) assert invariance of IPCA scores and singular values, and the text then asserts that the set of m points with smallest scaled distance inside the bounding box of Eq. (3) is identical for Z and ZA+1v^T. This requires a proof that the selection-box rule itself is orthogonal equivariant, i.e., that the expanded bounding box in the transformed IPCA coordinates contains exactly the transformed members of the original candidate set. The proof also presupposes that this candidate set has at least m elements. These points are not established, and the arbitrary 0.5 expansion factor is not justified. Please supply a lemma for the box rule and address existence before invoking induction for H^(k).
  3. [Algorithm 1, Section 2.2] The stopping rule is underspecified when h = alpha*n is not a multiple of the batch size m. The loop 'for k = 1 to floor(h/m - 1)' adds m points per iteration, so the final selected set has size m*floor(h/m), which can be strictly less than h. The text describes the target as h = alpha*n points, but no rounding rule or partial final batch is defined. This affects the actual subset used for location and covariance, and should be clarified (e.g., by defining h as the integer multiple of m closest to alpha*n, or by allowing a final partial batch).
  4. [Section 2.2, Eq. (1), Algorithm 1 Step 1] The orthogonality proof treats projection depth as exactly affine equivariant via Eq. (1), but the algorithm states that u is randomly sampled from the unit p-sphere. A finite random draw of directions is not exactly orthogonal equivariant under a fixed draw, so the exact equivariance claimed in Theorem 1 does not hold for the implemented Algorithm 1. Please state whether the theorem applies to the population depth with a supremum over all directions, and if the random sampling is retained, clarify the sense in which equivariance holds (e.g., in distribution, or for direction sets transformed by A).
minor comments (6)
  1. [Eq. (3)] The term delta_j^(k) is used but never defined; the sentence following the equation should define delta_j^(k) = 0.5 (max - min) and the displayed formula needs correction.
  2. [Eq. (6)] The variance formula 'sigma = diag(L)^2' appears to be a typo; if sigma denotes standard deviations, it should be sqrt(diag(L)), and if it denotes variances, the notation should be consistent with the use in Eq. (8).
  3. [Section 5.2] The execution time '11.43.03 ms' contains a typo.
  4. [Figures 7, 8, 10 captions] The captions list 'CPCA, DetMCD-PCA, FIR-PCA, and FIR-PCA'; the second FIR-PCA should be FDB-PCA.
  5. [Section 3.3] The complexity derivation would be clearer if it stated kappa = h/m - 1 before the equality O(kappa^2 m p + kappa m p^2) = O(kappa n p + n p^2).
  6. [Theorem 1] The theorem calls Z 'nonsingular' with n observations and p features; for rectangular matrices, the intended condition (e.g., full column rank) should be stated.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: FIR-PCA's estimates follow from an explicit iterative subset-selection rule; no target output is used as input and no load-bearing self-citation appears.

full rationale

The derivation chain is self-contained. Algorithm 1 selects a subset via projection depth and IPCA-based scaled distances without using the final robust PCA scores, covariance, or outlier labels as inputs. The robust location and covariance are then computed directly from the selected subset (Algorithm 1 step 4), and FIR-PCA's scores, principal vectors, variance, and outlier distances are deterministic functions of that estimate. No parameter is fitted to the known-outlier ground truth or to the reported benchmark outcomes: m and alpha are user-specified, and the 0.5 box-expansion factor is a fixed algorithmic constant rather than a fitted quantity. External results (Zuo & Serfling 2000; Ross et al. 2008) are cited for projection depth and IPCA, and neither citation is authored by the present paper's authors nor does it smuggle in FIR's central claim. The main caveats are correctness issues rather than circularity: Algorithm 1 step 3(d) has no stated fallback if fewer than m unselected points lie in the expanded selection box, and Theorem 1's invariance proof presupposes that the selected set exists and is identical under orthogonal transformation. Section 6 also concedes that p > n and partial-outlier cases are out of scope. These gaps do not make the claimed output equivalent to its input by construction, so no circular step is exhibited.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central claim rests on the reliability of the iterative selection heuristic, which depends on several user-chosen constants (alpha, m, expansion factor 0.5) and on an unproved equivariance of the selection rule. No new entities are postulated.

free parameters (4)
  • alpha (proportion of data selected) = 0.75 for 10% outliers, 0.5 for 40% outliers
    User-specified parameter controlling the breakdown point; set differently for different contamination levels in Section 4.1, affecting all reported results.
  • batch size m = not reported for main experiments
    Number of points added per iteration; Table 3 shows m=10,40,60,120,240,300 for different n, but the main comparisons do not state m.
  • Selection box expansion factor = 0.5
    Ad hoc constant in Equation 3 used to expand the bounding box; no justification or sensitivity analysis is given.
  • Number of random projection directions for depth (tau) = unspecified
    Projection depth in Equation 1 requires a supremum over directions, approximated by random sampling; the number of directions affects the initial m points and is never reported.
assumptions (4)
  • standard math Projection depth is affine equivariant (Zuo and Serfling 2000).
    Used in Theorem 1 to claim the initial index set is equivariant; treated as an existing result without proof.
  • standard math Incremental PCA correctly updates SVD incrementally and is orthogonal equivariant.
    The algorithm relies on IPCA projections and singular values being correct and equivariant under orthogonal transformations; asserted in the proof without demonstration.
  • ad hoc to paper The 0.5-expansion selection box rule is orthogonal equivariant.
    The proof of Theorem 1 implicitly assumes the bounding box expanded by a factor of 0.5 transforms correctly under rotation and translation; this is asserted, not proven.
  • domain assumption Inlier distribution is captured by Mahalanobis-like scaled distances and a two-dimensional selection box.
    The selection rule assumes the IPCA projection and the leading two singular vectors carry enough information to separate inliers from outliers; no theoretical guarantee for arbitrary or non-elliptical data.

how reviews work

0 comments
Cite this review

Pith. "Pith review of A Fast Iterative Robust Principal Component Analysis Method." pith.science (2026). https://pith.science/paper/TNAXRRF6

@misc{pith2026250616013,
  author       = {Pith},
  title        = {Pith review of: A Fast Iterative Robust Principal Component Analysis Method},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TNAXRRF6}},
  note         = {Machine review of arXiv:2506.16013}
}
read the original abstract

Principal Component Analysis (PCA) is widely used for dimensionality reduction and data analysis. However, PCA results are adversely affected by outliers often observed in real-world data. Existing robust PCA methods are often computationally expensive or exhibit limited robustness. In this work, we introduce a Fast Iterative Robust (FIR) PCA method by efficiently estimating the inliers center location and covariance. Our approach leverages Incremental PCA (IPCA) to iteratively construct a subset of data points that ensures improved location and covariance estimation that effectively mitigates the influence of outliers on PCA projection. We demonstrate that our method achieves competitive accuracy and performance compared to existing robust location and covariance methods while offering improved robustness to outlier contamination. We utilize simulated and real-world datasets to evaluate and demonstrate the efficacy of our approach in identifying and preserving underlying data structures in the presence of contamination.

Figures

Figures reproduced from arXiv: 2506.16013 by the authors.

Figure 1
Figure 1. Overview of FIR pipeline. 2002). Instead of requiring the entire dataset to be loaded into memory, IPCA updates the singular value decomposition (SVD) as new data arrive. This memory-efficient benefit makes the IPCA particularly attractive for large datasets and streaming data (Sun & Liu 2011, Yin et al. 2015, Yue et al. 2022, Zhao et al. 2006). Given a new batch of data, the SVD update in IPCA consist of adjusting … view at source ↗
Figure 2
Figure 2. Selected points with DetMCD, FDB, and FIR. The example uses [PITH_FULL_IMAGE:figures/full_fig_p013_2.png] view at source ↗
Figure 3
Figure 3. Approximation errors with varying number of features [PITH_FULL_IMAGE:figures/full_fig_p019_3.png] view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: Approximation errors with varying outlier percentages. The example uses the [PITH_FULL_IMAGE:figures/full_fig_p019_4.png]
Figure 5
Figure 5. Figure 5: CPCA, DetMCD-PCA, FDB-PCA, and FIR-PCA projections of [PITH_FULL_IMAGE:figures/full_fig_p020_5.png]
Figure 6
Figure 6. Figure 6: Execution time in s of DetMCD, FDB, and FIR for varying n with p = 5, 40, and varying p with n = 1000. The bottom row shows a visualization of the top row without DetMCD. 21 [PITH_FULL_IMAGE:figures/full_fig_p021_6.png]
Figure 7
Figure 7. Figure 7: Score (top row) and outlier maps (bottom row) for CPCA, DetMCD-PCA, FIR [PITH_FULL_IMAGE:figures/full_fig_p023_7.png]
Figure 8
Figure 8. Figure 8: Score (top row) and outlier maps (bottom row) for CPCA, DetcMCD-PCA, FIR [PITH_FULL_IMAGE:figures/full_fig_p024_8.png]
Figure 9
Figure 9. Figure 9: In Figure 9a, 117 is closer to the distribution, while in the remaining robust PCA [PITH_FULL_IMAGE:figures/full_fig_p025_9.png]
Figure 10
Figure 10. Figure 10: Outlier maps for CPCA, DetMCD-PCA, FIR-PCA, and FIR-PCA. The red [PITH_FULL_IMAGE:figures/full_fig_p026_10.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Robust Heteroskedastic Matrix Factorization: A Generalization of PCA that Flags Outliers and Handles Missing Data

    astro-ph.IM 2026-07 conditional novelty 5.0 of 10

    A robust, heteroskedastic matrix factorization method generalizes PCA to handle per-feature uncertainties, missing data, and outlier detection via Student-t likelihood iterative reweighting.

Reference graph

Works this paper leans on

7 extracted references · 5 canonical work pages · cited by 1 Pith paper

  1. [1]

    , " * write output.state after.block = add.period write newline

    ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.all := #1 'mid.sentence := #2 '...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in " " * FUNCTION format....

  3. [3]

    , " * write output.state after.block = add.period write newline

    ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.all := #1 'mid.sentence := #2 '...

    work page

  4. [4]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in " " * FUNCTION format....

    work page

  5. [5]

    , " * write output.state after.block = add.period write newline

    ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type url volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.all := #1 'mid.sentence := ...

    work page

  6. [6]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in " " * FUNCTION format....

    work page

  7. [7]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in " " * FUNCTION format....

    work page

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.