Pith. sign in

REVIEW 3 major objections 5 minor 32 references

ESS-ReduNet: Enhancing Subspace Separability of ReduNet via Dynamic Expansion with Bayesian Inference

T0 review · 3 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Dynamic expansion weighting plus Bayesian membership correction makes ReduNet converge in about a tenth of the layers and yields better-separated, more classifiable features.

desk verdict Promising plug-in fixes for ReduNet, but the headline speedup relies on an unfair stopping-rule comparison and needs re-benchmarking. read the letter →

arxiv 2411.17961 v1 pith:WW6PHLUS submitted 2024-11-27 cs.LG

Xiaojie Yu , Haibo Zhang , Lizhi Peng , Fengyang Sun , Jeremiah Deng This is my paper · ORCID
classification cs.LG MSC 68T0762F15
keywords ReduNetmaximalcodingratereductionsubspacelearningBayesianinferencerepresentationearlystoppingconditionnumbermembershipestimation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that ReduNet, the white-box network built from maximal coding rate reduction, can be made both faster and more accurate by fixing two sources of error: the limited span of the feature space and the unreliable membership estimates used in feature updates. The proposed ESS-ReduNet dynamically increases the weight of the expansion operator, the term that moves features into the complement of the spanned space, whenever the estimation function misclassifies samples, and it uses Bayesian inference to blend label information into the membership estimates in a way that can be reused at test time. It also adds a condition-number stability check as an early-stopping rule. If the paper is right, ReduNet becomes practical, needing roughly ten times fewer layers to converge while producing transformed features that are better separated, with SVM accuracy rising from 0.65 to 0.96 on the ESR dataset.

What carries the argument

The central object is the per-layer ReduNet update $z_{\ell+1}\propto z_\ell+\eta(E_\ell z_\ell-\sum_j\gamma_j C_\ell^j z_\ell \hat{\pi}_j(z_\ell))$, where $E_\ell=\alpha(I+\alpha Z_\ell Z_\ell^T)^{-1}$ is the expansion operator (shown to be approximately the ridge-regression residual, i.e., projection onto the complement of the spanned space) and $C_\ell^j$ are compression operators. ESS-ReduNet's mechanism is the weighted expansion $w(\tau_\ell)E_\ell z_\ell$ with $w(\tau)=\min(\exp(\tau),u)$ and $\tau$ incremented by 0.1 on every layer with estimation errors, combined with Bayesian-corrected memberships $p_c^i(z_\ell)$ defined through the confusion posterior $P(z\in C_i\mid z\to C_j)$. A condition number $\sigma_{\max}/\sigma_{\min}$ of the matrices in $E_\ell$ and $C_\ell^j$ serves as an auxiliary early-stopping signal. Together these pieces enlarge the overall spanned space so class subspaces decouple, and correct the membership estimates so samples move toward the right subspace.

What would settle it

Train ESS-ReduNet on a dataset whose intrinsic dimension equals the lifted feature dimension, leaving no orthogonal complement for the expansion operator to enlarge the span; if the estimation-function errors still vanish and convergence remains rapid, the expansion schedule is not the operative mechanism, whereas a stall would confirm the paper's causal story.

Watch

Extended reading notes

Core claim

ReduNet builds each layer's parameters from the features of the previous layer, updating features by a gradient step of a maximal coding rate reduction objective. Its expansion operator $E_\ell$ pushes features into the complement of the spanned space, and its compression operators $C_\ell^j$ pull samples toward the subspace of class $j$, using estimated memberships instead of labels. ESS-ReduNet modifies this update in two ways: it multiplies the expansion term by a truncated exponential weight $w(\tau_\ell)=\min(\exp(\tau_\ell),u)$ that grows whenever the estimation function misclassifies a sample, and it replaces the raw membership estimate with a Bayesian posterior $p_c^i(z)=\sum_j P(z\in C_i\mid z\to C_j)\,\hat{\pi}_j(z)$ that is computed during training by comparing estimates with labels and then reused at test time. The paper reports that this joint modification reduces the number of layers needed for convergence from over 3000 to roughly 100 to 300 on four benchmark datasets, and on the ESR dataset it reduces misclassified membership estimates to zero by layer 19 while lifting SVM accuracy on the transformed features from 0.65 to 0.96.

Load-bearing premise

The claimed speedup rests on the unproved assumption that the hand-tuned rule for strengthening the expansion term, increasing its weight by a fixed amount every time any sample is misclassified, reliably separates class subspaces and never steers feature updates in the wrong direction, on any dataset.

Editorial extensions

If this is right

  • ReduNet training becomes practical: instead of thousands of layers, a few hundred suffice for convergence on standard benchmarks, cutting the compute and storage of parameters.
  • The plug-in modules, weighted expansion and Bayesian correction, carry over to the Fourier version of ReduNet and improve its accuracy as well.
  • The condition-number halting rule gives a cheap stopping signal that prevents the feature-quality degradation seen when a poorly constructed network trains too long.
  • Downstream classifiers such as SVM, KNN, and NSC benefit from the more orthogonalized features, with large gains on datasets that were previously challenging, including the ESR SVM accuracy jump from 0.65 to 0.96.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The Bayesian posterior effectively converts training-time estimation errors into a confusion matrix, so ESS-ReduNet behaves like a self-training procedure that injects label information only when the unsupervised estimates are unreliable; the same correction could apply to other iterative subspace-clustering or self-supervised methods with soft memberships.
  • The fixed schedule $\tau_{\ell+1}=\tau_\ell+0.1$ is a single global knob; on datasets with more classes or different scales, the same increment may overshoot or undershoot, so a dataset-dependent schedule driven by the observed misclassification rate is a natural testable extension.
  • The geometric reading of the expansion operator as a residual projection suggests a convergence proof could be attempted under a margin condition on the class subspaces; the paper does not provide one, leaving a formal guarantee open.
  • Because the condition number stabilizes earlier than the objective function, this stopping heuristic could serve as a general diagnostic for layer-wise-constructed white-box networks beyond ReduNet.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. This paper proposes ESS-ReduNet, a modification of the white-box ReduNet architecture that aims to improve subspace separability and accelerate convergence. Two main mechanisms are introduced: a Bayesian-inference correction of the class-membership estimates used in the layer-wise updates (so that label information can be reused at test time without inconsistency), and a dynamically weighted expansion operator that is strengthened when the training appears to stagnate. A condition-number-based stopping rule is also proposed as an auxiliary convergence criterion. The paper reports experiments on seven datasets, claiming more than a 10-fold reduction in the number of layers needed for convergence on four of them and large gains in SVM/KNN/NSC accuracy of the transformed features (e.g., 0.65 to 0.96 SVM accuracy on ESR). An ablation study indicates that both the Bayesian module and the dynamic expansion module contribute to the improvement.

Significance. If the reported gains are robust, the paper would make ReduNet training substantially more practical and would provide a useful case study in combining white-box, layer-wise constructed networks with label-derived Bayesian corrections. The geometric interpretation of the expansion operator (Section 4.2 and Appendix A) is a coherent and useful addition, and the ablation in Figure 9 provides evidence that both proposed modules are active ingredients. However, the central quantitative claims (10x layer reduction and 47% accuracy gain) are currently supported only under a comparison protocol that does not apply matched stopping rules to the baseline, and the hand-crafted expansion schedule is not analyzed for sensitivity. The paper does not release code or report multiple seeds, which limits the reproducibility of the empirical claims.

major comments (3)
  1. [Section 5.3, Table 1; Section 3, Figure 3d] The comparison between ESS-ReduNet and ReduNet is not made under matched stopping conditions. On the ESR dataset, Figure 3d shows ReduNet's condition number stabilizing around layer 1600, and the text in Section 3 explicitly says that further training of a poorly constructed network degrades feature quality. Yet Table 1 reports ReduNet's convergence layer as '>3000' and evaluates its SVM accuracy (0.65) at that over-run endpoint, while ESS-ReduNet is stopped at layer 199 by the condition-number criterion. This mismatched protocol inflates both the '>10x speedup' and the '47% SVM accuracy gain' claimed in the abstract. The authors should report ReduNet's accuracy at its own condition-number plateau (or at the same layer count as ESS-ReduNet) and apply the identical stopping rule to both methods across all datasets.
  2. [Section 4.2, Eq. (11); Algorithm 1 lines 10-13] The expansion schedule w(tau)=min(exp(tau),u) with tau incremented by 0.1 per error layer introduces two free parameters (the increment and the cap u) that are fixed without sensitivity analysis. The paper asserts that this schedule robustly enlarges the spanned space to decouple class subspaces, but no ablation over these parameters is given, and no convergence guarantee is provided. To support the claim that the method is not overly sensitive to hand-tuned dynamics, the authors should report results for at least a few values of the increment and cap, or justify their chosen values with a principled derivation.
  3. [Section 5 (all experiments)] All experimental results appear to be single runs with no error bars, multiple seeds, or significance testing, and no code is released. Since the paper's central claims are quantitative (layer reduction and accuracy improvement), the authors should release code and report statistics over multiple random initializations and train/test splits, or justify why single runs are sufficient for these claims.
minor comments (5)
  1. [Appendix A] The phrase 'spannd space' appears; it should be 'spanned space'.
  2. [Section 2.2] The word 'compresssion' should be 'compression' in the sentence about white-box transformers.
  3. [Figure 9a] The legend entries 'Bayes_1' through 'Bayes_9' and 'ReduNet_1' through 'ReduNet_9' are not explained in the text; the caption should state that they correspond to different channel numbers Nc.
  4. [Section 4.1, Eq. (7)] The subscripts in the posterior notation p_ij and P(z in C_i | z -> C_j) are not clearly defined; please specify the direction of the conditional (i.e., which index corresponds to the observed class and which to the true class) in the text.
  5. [Section 4.3] The stopping criterion 'no change on condition number' is not quantified; the authors should define the tolerance and the number of layers over which the change is checked, so that the stopping rule is reproducible.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the ESS-ReduNet updates are explicit supervised modifications of ReduNet, and the reported gains are empirical. The unmatched stopping rule is an experimental fairness concern, not a circular derivation.

full rationale

ESS-ReduNet's core updates (Algorithm 1 lines 10-13) combine a hand-set expansion weight w(τ)=min(exp(τ),u) with a Bayesian-corrected membership estimate p_c. These are defined independently of the downstream metrics (SVM/KNN/NSC accuracy, layer counts), so no output is fed back to define an input. The Bayesian correction (Eqs. 7-9) is a calibration of the estimator using training labels; using labels in training and then evaluating on the same task is standard supervised practice, not circular. The geometric reading of E_ℓ z_ℓ as a ridge-regression residual (Eqs. 10 and 12) is a standard Woodbury identity, not a self-referential derivation. Citations to Chan et al. (2022) supply the external ReduNet baseline; the cited Yu et al. (2020) concerns evaluation protocols; none of the load-bearing cited results are authored by the present paper's authors. One experimental-fairness issue should be distinguished from circularity: Section 3 says ReduNet's condition number 'stabilizes around the 1600th layer', yet Table 1 reports ReduNet's convergence layer as '> 3000' while ESS-ReduNet stops at 199. This may inflate the '>10x' speedup and the 47% SVM gain, but the layer counts are chosen evaluation endpoints rather than equation-level reductions of the claimed result to its inputs. Hence no significant circularity.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The central claim depends on ReduNet's MCR2 framework, one hand-crafted expansion schedule, a label-derived posterior that must transfer to test data, and an undocumented condition-number stopping rule. No new entities are introduced.

free parameters (6)
  • Expansion weight cap u = 10
    Equation (11) caps w(τ) at u; u=10 is chosen so that η·u does not exceed 1, with no sensitivity analysis.
  • Expansion increment Δτ = 0.1
    Algorithm 1 line 13 increments τ by 0.1 each layer that has estimation errors; this schedule is arbitrary and untested.
  • Condition number stopping tolerance
    Algorithm 1 line 19 stops when there is 'no change' in condition number; no threshold is defined, yet this rule determines every Layer value in Table 1.
  • Estimation function parameter λ
    Equation (6) defines π_hat with a parameter λ, but no value or tuning strategy is reported anywhere.
  • Channel number Nc
    The lifting-operation channel count is varied in Figure 9 but is not stated for the main experiments reported in Table 1.
  • Learning rate η and distortion ε² = η=0.1, ε²=0.1
    These are fixed for all datasets; they may be inherited from ReduNet conventions, but they are treated as constants without tuning or justification.
assumptions (5)
  • domain assumption ReduNet's MCR2 objective and layer construction produce useful representations.
    The paper adopts the rate-reduction framework from Chan et al. 2022 as its starting point and does not re-derive its optimality.
  • ad hoc to paper Increasing the weight of the expansion operator improves subspace separability and estimation accuracy.
    Equation 11 and Algorithm 1 line 10 introduce the schedule; it is motivated by the geometric interpretation of E_l z_l but has no theorem guaranteeing convergence or stability.
  • domain assumption The posterior confusion matrix estimated on training labels at each layer is a valid correction for test samples.
    Equations 7-9 assume the likelihoods P(z→Cj|z∈Ci), computed from training data, carry over to test data despite the network changing layer by layer.
  • domain assumption Condition-number stability is a safe stopping criterion.
    Section 4.3 treats small, stable condition numbers as evidence of convergence; the connection to final classification accuracy is empirical only.
  • standard math The Woodbury identity and the ridge-regression residual form in Appendix A are valid.
    Used to connect E_l z_l to projection residuals; this step is standard and unlikely to be the failure point.

how reviews work

0 comments
Cite this review

Pith. "Pith review of ESS-ReduNet: Enhancing Subspace Separability of ReduNet via Dynamic Expansion with Bayesian Inference." pith.science (2026). https://pith.science/paper/WW6PHLUS

@misc{pith2026241117961,
  author       = {Pith},
  title        = {Pith review of: ESS-ReduNet: Enhancing Subspace Separability of ReduNet via Dynamic Expansion with Bayesian Inference},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/WW6PHLUS}},
  note         = {Machine review of arXiv:2411.17961}
}
read the original abstract

ReduNet is a deep neural network model that leverages the principle of maximal coding rate \textbf{redu}ction to transform original data samples into a low-dimensional, linear discriminative feature representation. Unlike traditional deep learning frameworks, ReduNet constructs its parameters explicitly layer by layer, with each layer's parameters derived based on the features transformed from the preceding layer. Rather than directly using labels, ReduNet uses the similarity between each category's spanned subspace and the data samples for feature updates at each layer. This may lead to features being updated in the wrong direction, impairing the correct construction of network parameters and reducing the network's convergence speed. To address this issue, based on the geometric interpretation of the network parameters, this paper presents ESS-ReduNet to enhance the separability of each category's subspace by dynamically controlling the expansion of the overall spanned space of the samples. Meanwhile, label knowledge is incorporated with Bayesian inference to encourage the decoupling of subspaces. Finally, stability, as assessed by the condition number, serves as an auxiliary criterion for halting training. Experiments on the ESR, HAR, Covertype, and Gas datasets demonstrate that ESS-ReduNet achieves more than 10x improvement in convergence compared to ReduNet. Notably, on the ESR dataset, the features transformed by ESS-ReduNet achieve a 47\% improvement in SVM classification accuracy.

Figures

Figures reproduced from arXiv: 2411.17961 by the authors.

Figure 1
Figure 1. The Vicious Cycle of ReduNet. S0 and S1 are spanned subspaces of class 0 and class 1, respectively. z → S0 means sample z updating towards S0. The inaccuracy of the estimation arises because the separa￾bility of subspaces may be compromised due to the limited overall spanned space. The constrained and entangled sub￾spaces may fail to accurately estimate the degree of member￾ship effectively. As illustrated in [PITH… view at source ↗
Figure 2
Figure 2. A Layer of ReduNet Here, Eℓ = α(I + αZℓZ T ℓ ) −1 ∈ R n×n and C j ℓ = αj (I + αjZℓΠjZ T ℓ ) −1 ∈ R n×n. Thereby, for a feature zℓ, the in￾crement transform g(·, θℓ) on the ℓ-th layer is defined as: zℓ+1 ∝ zℓ + η · g(zℓ, θℓ) subject to zℓ+1 ∈ S n−1 . (4) g(zℓ, θℓ) = Eℓzℓ − X k j=1 γjC j ℓ zℓπˆ j (zℓ) ∈ R n . (5) Note that unlike Eq.3, Eq.5 uses estimation functions {πˆ j } k j=1 for feature updates instead of directl… view at source ↗
Figure 4
Figure 4. Overview of ESS-ReduNet computational resources and maintain feature quality. 4 ESS-ReduNet Three questions need to be addressed: 1. Can we introduce labels to guide feature updates in a relatively correct direction without causing the inconsis￾tency issue? 2. Besides using labels, are there other methods to improve the accuracy of estimation functions? 3. Are there other metrics that can be used to assist in de￾ter… view at source ↗
Figures from the paper (22 more)
Figure 3
Figure 3. Figure 3: (a): The number of misclassified labels of [PITH_FULL_IMAGE:figures/full_fig_p004_3.png]
Figure 5
Figure 5. Figure 5: The Geometric Interpretation of Least Squares [PITH_FULL_IMAGE:figures/full_fig_p005_5.png]
Figure 6
Figure 6. Figure 6: Performance of ESS-ReduNet on ESR Dataset. [PITH_FULL_IMAGE:figures/full_fig_p006_6.png]
Figure 7
Figure 7. Figure 7: Visualization of ESR Feature Orthogonalization at the [PITH_FULL_IMAGE:figures/full_fig_p006_7.png]
Figure 8
Figure 8. Figure 8: Performance Comparison on the Fourier Version. [PITH_FULL_IMAGE:figures/full_fig_p007_8.png]
Figure 9
Figure 9. Figure 9: (a):The misclassification number of estimation functions [PITH_FULL_IMAGE:figures/full_fig_p007_9.png]
Figure 10
Figure 10. Figure 10: The Geometric Interpretation of Least Squares [PITH_FULL_IMAGE:figures/full_fig_p009_10.png]
Figure 13
Figure 13. Figure 13: HAR 0 50 100 150 200 Layer 2.85 2.90 2.95 3.00 3.05 3.10 3.15 Number of misclassified labels Min: 3 (a) ReduNet 0 50 100 150 200 Layer 0.0 0.5 1.0 1.5 2.0 2.5 3.0 Number of misclassified labels Min: 0 (b) ESS-ReduNet [PITH_FULL_IMAGE:figures/full_fig_p010_13.png]
Figure 14
Figure 14. Figure 14: mfeatFactors D.3 Rank Trend Figures 23 to 28 show the changes in rank across six datasets. It is evident that the features transformed by ESS-ReduNet have a larger overall spanned space. Moreover, there is no oc￾currence of the overall spanned space collapsing, as dep…
Figure 11
Figure 11. Figure 11: covtype 0 1000 2000 3000 Layer 470 480 490 Number of misclassified labels Min: 462 (a) ReduNet 0 1000 2000 3000 Layer 100 200 300 400 500 Number of misclassified labels Min: 89 (b) ESS-ReduNet [PITH_FULL_IMAGE:figures/full_fig_p010_11.png]
Figure 12
Figure 12. Figure 12: gas D.2 Objective Function Curve Figures 17 to 22 depict the curve of objective functions in six data sets, where ESS-ReduNet achieves higher values of MCR2 . 0 1000 2000 3000 Layer 400 500 600 700 800 Number of misclassified labels Min: 415 (a) ReduNet 0 1000 2000 30…
Figure 20
Figure 20. Figure 20: mfeatFactors 0 50 100 150 200 Layer 200 400 600 800 Loss Total Expand Compress (a) ReduNet 0 50 100 150 200 Layer 0 200 400 600 800 Loss Total Expand Compress (b) ESS-ReduNet [PITH_FULL_IMAGE:figures/full_fig_p011_20.png]
Figure 21
Figure 21. Figure 21: mfeatFourier 0 50 100 150 200 Layer 0 200 400 600 800 1000 1200 Loss Total Expand Compress (a) ReduNet 0 50 100 150 200 Layer 0 500 1000 1500 Loss Total Expand Compress (b) ESS-ReduNet [PITH_FULL_IMAGE:figures/full_fig_p011_21.png]
Figure 22
Figure 22. Figure 22: musk 0 1000 2000 3000 Layer 100 110 120 130 140 Rank All_data Class0 Class1 Class2 Class3 Class4 Class5 Class6 (a) ReduNet 0 1000 2000 3000 Layer 100 200 300 400 500 Rank All_data Class0 Class1 Class2 Class3 Class4 Class5 Class6 (b) ESS-ReduNet [PITH_FULL_IMAGE:figur…
Figure 23
Figure 23. Figure 23: covtype [PITH_FULL_IMAGE:figures/full_fig_p011_23.png]
Figure 24
Figure 24. Figure 24: gas 0 100 200 300 400 500 Layer 0 200 400 600 800 1000 1200 Rank All_data Class0 Class1 Class2 Class3 Class4 Class5 (a) ReduNet 0 100 200 300 400 500 Layer 200 400 600 800 1000 1200 Rank All_data Class0 Class1 Class2 Class3 Class4 Class5 (b) ESS-ReduNet [PITH_FULL_IM…
Figure 25
Figure 25. Figure 25: HARv1 0 1000 2000 3000 Layer 0 200 400 600 800 Rank All_data Class0 Class1 Class2 Class3 Class4 Class5 Class6 Class7 Class8 Class9 (a) ReduNet 0 1000 2000 3000 Layer 200 400 600 800 1000 Rank All_data Class0 Class1 Class2 Class3 Class4 Class5 Class6 Class7 Class8 Clas…
Figure 30
Figure 30. Figure 30: gas 0 1000 2000 3000 Layer 0 2000 4000 6000 8000 10000 12000 Condition Number All_data Class0 Class1 Class2 Class3 Class4 Class5 (a) ReduNet 0 1000 2000 3000 Layer 0 2000 4000 6000 8000 10000 12000 Condition Number All_data Class0 Class1 Class2 Class3 Class4 Class5 (b…
Figure 31
Figure 31. Figure 31: HARv1 [PITH_FULL_IMAGE:figures/full_fig_p012_31.png]
Figure 32
Figure 32. Figure 32: mfeatFactors 0 50 100 150 200 Layer 0 2000 4000 6000 Condition Number All_data Class0 Class1 Class2 Class3 Class4 Class5 Class6 Class7 Class8 Class9 (a) ReduNet 0 50 100 150 200 Layer 0 2000 4000 6000 Condition Number All_data Class0 Class1 Class2 Class3 Class4 Class5…
Figure 33
Figure 33. Figure 33: mfeatFourier 0 50 100 150 200 Layer 0 2000 4000 6000 8000 10000 Condition Number All_data Class0 Class1 (a) ReduNet 0 50 100 150 200 Layer 0 2000 4000 6000 8000 10000 Condition Number All_data Class0 Class1 (b) ESS-ReduNet [PITH_FULL_IMAGE:figures/full_fig_p013_33.png]
Figure 34
Figure 34. Figure 34: musk [PITH_FULL_IMAGE:figures/full_fig_p013_34.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

32 extracted references · 25 canonical work pages

  1. [1]

    Covertype,

    [Blackard, 1998] Jock Blackard. Covertype,

    work page 1998

  2. [7]

    [Fang et al., 2021] Cong Fang, Hangfeng He, Qi Long, and Weijie J. Su. Exploring deep neural networks via layer- peeled model: Minority collapse in imbalanced train- ing. Proceedings of the National Academy of Sciences , 118(43):e2103091118, October

    work page 2021

  3. [10]

    [He et al., 2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun

    arXiv:2010.03697 [cs]. [He et al., 2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778,

    arXiv 2010

  4. [11]

    [Hinton and Salakhutdinov, 2006] G. E. Hinton and R. R. Salakhutdinov. Reducing the Dimensionality of Data with Neural Networks. Science, 313(5786):504–507, July

    work page 2006

  5. [12]

    Deep subspace clustering net- works

    [Ji et al., 2017] Pan Ji, Tong Zhang, Hongdong Li, Mathieu Salzmann, and Ian Reid. Deep subspace clustering net- works. Advances in neural information processing sys- tems, 30,

    work page 2017

  6. [15]

    Gradient-based learning ap- plied to document recognition

    [LeCun et al., 1998] Yann LeCun, L ´eon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning ap- plied to document recognition. Proceedings of the IEEE, 86(11):2278–2324,

    1998

  7. [16]

    [Lezama et al., 2018] Jos´e Lezama, Qiang Qiu, Pablo Mus´e, and Guillermo Sapiro

    Publisher: Ieee. [Lezama et al., 2018] Jos´e Lezama, Qiang Qiu, Pablo Mus´e, and Guillermo Sapiro. Ole: Orthogonal low-rank embedding-a plug and play geometric loss for deep learn- ing. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 8109–8118,

    work page 2018

  8. [20]

    [Qiuyi Wu, 2017] Ernest Fokoue Qiuyi Wu

    arXiv:1709.08374 [cs]. [Qiuyi Wu, 2017] Ernest Fokoue Qiuyi Wu. Epileptic Seizure Recognition,

    arXiv 2017

Show all 32 references
  1. [21]

    Linear algebra and its appli- cations

    [Strang, 2012] Gilbert Strang. Linear algebra and its appli- cations

    2012

  2. [22]

    Tabeart, Sarah L

    [Tabeart et al., 2019] Jemima M. Tabeart, Sarah L. Dance, Amos S. Lawless, Nancy K. Nichols, and Joanne A. Waller. Improving the condition number of estimated covariance matrices, October

    2019

  3. [24]

    van Wieringen

    [van Wieringen, 2023] Wessel N. van Wieringen. Lecture notes on ridge regression, June

    2023

  4. [25]

    [Vergara, 2012] Alexander Vergara

    arXiv:1509.09169 [stat]. [Vergara, 2012] Alexander Vergara. Gas Sensor Array Drift at Different Concentrations,

    2012 arXiv

  5. [26]

    Learning Diverse and Discriminative Representations via the Principle of Maxi- mal Coding Rate Reduction

    [Yu et al., 2020] Yaodong Yu, Kwan Ho Ryan Chan, Chong You, Chaobing Song, and Yi Ma. Learning Diverse and Discriminative Representations via the Principle of Maxi- mal Coding Rate Reduction. In H. Larochelle, M. Ranzato, R. Hadsell, M. F. Balcan, and H. Lin, editors, Advances...

    2020

  6. [27]

    [Yu et al., 2023b] Yaodong Yu, Sam Buchanan, Druv Pai, Tianzhe Chu, Ziyang Wu, Shengbang Tong, Benjamin Ha- effele, and Yi Ma

    arXiv:2311.13110 [cs]. [Yu et al., 2023b] Yaodong Yu, Sam Buchanan, Druv Pai, Tianzhe Chu, Ziyang Wu, Shengbang Tong, Benjamin Ha- effele, and Yi Ma. White-box transformers via sparse rate reduction. Advances in Neural Information Processing Systems, 36:9422–9457,

    arXiv

  7. [28]

    Scalable Deep k-Subspace Clustering

    [Zhang et al., 2019a] Tong Zhang, Pan Ji, Mehrtash Harandi, Richard Hartley, and Ian Reid. Scalable Deep k-Subspace Clustering. In C.V . Jawahar, Hongdong Li, Greg Mori, and Konrad Schindler, editors, Computer Vision – ACCV 2018, volume 11365, pages 466–481. Springer Interna- ...

    2018

  8. [29]

    Deep adversarial subspace clustering

    [Zhou et al., 2018] Pan Zhou, Yunqing Hou, and Jiashi Feng. Deep adversarial subspace clustering. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1596–1604,

    2018

  9. [30]

    A Ge- ometric Analysis of Neural Collapse with Unconstrained Features, May

    [Zhu et al., 2021] Zhihui Zhu, Tianyu Ding, Jinxin Zhou, Xiao Li, Chong You, Jeremias Sulam, and Qing Qu. A Ge- ometric Analysis of Neural Collapse with Unconstrained Features, May

    2021

  10. [31]

    arXiv:2105.02375 [cs, math, stat]. A Detailed Derivation of the Geometric Interpretation of Expansion Operators e=(I-P)b b left nullspace N(A*) column space C(A) perpendicular Pb Figure 10: The Geometric Interpretation of Least Squares Although Chan et al

    arXiv

  11. [32]

    For a least squares problem of Ax = b, the projection ma- trix is P = A(A∗A)−1A∗

    have discussed the relation- ship between expansion operators and ridge regression, we further clarify their geometric interpretation by comparing the forms of the expansion operators with those of least squares and ridge regression. For a least squares problem of Ax = b, the ...

    2012

  12. [1994]

    Multiple Features,

    [Duin, 1998] Robert Duin. Multiple Features,

    1998

  13. [1998]

    Re- duNet: A white-box deep network from the principle of maximizing rate reduction

    [Chan et al., 2022] Kwan Ho Ryan Chan, Yaodong Yu, Chong You, Haozhi Qi, John Wright, and Yi Ma. Re- duNet: A white-box deep network from the principle of maximizing rate reduction. The Journal of Machine Learning Research, 23(1):4907–5009,

    2022

  14. [2006]

    A U-turn on Double Descent: Rethinking Parameter Counting in Statistical Learning, October

    [Curth et al., 2023] Alicia Curth, Alan Jeffares, and Mihaela van der Schaar. A U-turn on Double Descent: Rethinking Parameter Counting in Statistical Learning, October

    2023

  15. [2007]

    [Papyan et al., 2020] Vardan Papyan, X. Y . Han, and David L. Donoho. Prevalence of neural collapse during the terminal phase of deep learning training. Proceedings of the National Academy of Sciences, 117(40):24652–24663, October

    2020

  16. [2012]

    [Krizhevsky et al., 2012] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. Imagenet classification with deep convolutional neural networks. Advances in neural infor- mation processing systems, 25,

    2012

  17. [2016]

    Haeffele, Chong You, and Ren´e Vidal

    [Haeffele et al., 2021] Benjamin D. Haeffele, Chong You, and Ren´e Vidal. A Critique of Self-Expressive Deep Sub- space Clustering, March

    2021

  18. [2017]

    Human Activity Recognition Using Smartphones,

    [Jorge Reyes-Ortiz, 2012] Davide Anguita Jorge Reyes- Ortiz. Human Activity Recognition Using Smartphones,

    2012

  19. [2018]

    Segmentation of Multivariate Mixed Data via Lossy Data Coding and Compression

    [Ma et al., 2007] Yi Ma, Harm Derksen, Wei Hong, and John Wright. Segmentation of Multivariate Mixed Data via Lossy Data Coding and Compression. IEEE Trans- actions on Pattern Analysis and Machine Intelligence , 29(9):1546–1562, September

    2007

  20. [2019]

    [Ur-Rashid, 2018] Harun Ur-Rashid

    arXiv:1810.10984 [math, stat]. [Ur-Rashid, 2018] Harun Ur-Rashid. Epileptic Seizure Recognition,

    2018 arXiv

  21. [2020]

    Deep Sparse Subspace Clustering, September

    [Peng et al., 2017] Xi Peng, Jiashi Feng, Shijie Xiao, Jiwen Lu, Zhang Yi, and Shuicheng Yan. Deep Sparse Subspace Clustering, September

    2017

  22. [2021]

    [Goodfellow et al., 2016] Ian Goodfellow, Yoshua Bengio, and Aaron Courville

    Publisher: Proceed- ings of the National Academy of Sciences. [Goodfellow et al., 2016] Ian Goodfellow, Yoshua Bengio, and Aaron Courville. Deep learning. MIT press,

    2016

  23. [2022]

    [Cover and Thomas, 2006] Thomas M

    Publisher: JMLRORG. [Cover and Thomas, 2006] Thomas M. Cover and Joy A. Thomas. Elements of Information Theory . Wiley, July

    2006

  24. [2023]

    [David Chapman, 1994] Ajay Jain David Chapman

    arXiv:2310.18988 [cs, stat]. [David Chapman, 1994] Ajay Jain David Chapman. Musk (Version 2),

    1994 arXiv

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.