{"id":"46ce0da8-fdc6-4da9-b1eb-0828d114963a","arxiv_id":"1908.09419","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"DCFSC replaces the trainable N by N self-expressive layer in deep subspace clustering with a closed-form matrix computed from latent features, cutting parameters dramatically while giving comparable results on small data and a strong result on COIL-100.","lead":"This paper removes the trainable self-expressive layer from deep subspace clustering and computes it with a closed-form matrix formula from the recommendation literature. The formula is already known from the EDSC baseline the paper cites, and the COIL-100 win comes with a deeper network, so the gain is partly architectural.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Stop-gradient closed-form self-expression is not shown to drive feature learning; with N≫d, B can be nearly identity, so the COIL-100 result may not validate subspace clustering.","rationale":"I agree with the CONDITIONAL verdict. The reader's weakest_assumption correctly identifies the stop-gradient alternation, but I want to sharpen it: the failure mode is not that B becomes stale between iterations (it is recomputed and is always the minimizer of Eq. 4 for the current features), but that the autoencoder is never asked to minimize the self-expression term, so there is no common objective being minimized. In the N≫d regime the self-expression equation is underdetermined and generically admits near-exact solutions, so B can be close to identity-like for almost any latent features; the clustering must then come from the spectral structure of an inverse-covariance affinity rather than from a learned union-of-subspaces representation. That makes the COIL-100 headline dependent on an unstated assumption about the learned features. The paper's exact closed-form derivation is correct and the memory comparison is meaningful, and the missing EDSC closed-form reference is a separate novelty issue. Because a targeted ablation could settle whether the mechanism is operative, CONDITIONAL is the right verdict and I do not propose changing it.","tokens_in":11696,"tokens_out":17763,"duration_ms":204570,"concrete_test":"Run a three-arm ablation on COIL-100 with the same DCFSC architecture: (A) the original Algorithm 2; (B) the same algorithm with an added self-expression loss γ||Z-BZ||_F^2 to the AE update (B detached), so both blocks minimize a common objective; (C) a plain autoencoder trained with Eq. 1 on Z directly, followed by spectral clustering on a kNN affinity in Z. If (A) does not clearly beat (C), and (B) does not meaningfully alter the error, then the stop-gradient B is not the load-bearing component and the method reduces to a vanilla AE plus a fixed affinity. Additionally, report final self-expression error and the fraction of |B| mass within vs across ground-truth clusters; if that fraction is near chance, the subspace interpretation fails.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Algorithm 2's load-bearing step is line 7: after computing B as the exact minimizer of Eq. 4 for the current latent Z, the gradient is stopped and Θ is updated using only the reconstruction loss (Eq. 1). This is not coordinate descent on a single objective: the B-update minimizes ||Z-BZ||_F^2 + λ||B||_F^2, but the AE-update has no self-expression term and treats B as constant, so a fixed point of the alternation is not necessarily a stationary point of any joint loss, and the paper gives no convergence argument. The issue is compounded by dimensionality: on COIL-100 N=7,200 while the latent dimension from the five-layer encoder is far smaller, so the linear system Z=BZ with diag(B)=0 is generically solvable almost exactly for arbitrary Z. Hence B is a near-identity linear map on the data span, and the only clustering signal is whatever affinity structure remains in B after spectral clustering. Nothing in the AE update explicitly rewards block-diagonal or union-of-subspaces structure, so the experiments do not establish that the closed-form self-expressive layer is the operative mechanism.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Deep Closed-Form Subspace Clustering (DCFSC), a variant of deep subspace clustering (DSC) in which the trainable N×N self-expressive layer is replaced by a matrix B computed in closed form from the current latent features by solving Eq. (4) with a Frobenius-norm regularizer and a zero-diagonal constraint. B has no trainable parameters, and training alternates between recomputing B from the current latent features (with the gradient stopped) and updating the autoencoder on the reconstruction loss only. The method is evaluated on Extended Yale B, ORL, and COIL-100; the paper reports clustering error rates and claims a state-of-the-art result on COIL-100 while using 0.16% of the parameters of DSC.","tokens_in":11967,"tokens_out":8134,"duration_ms":81339,"significance":"The memory-efficiency motivation is significant and timely: eliminating the N×N trainable self-expressive matrix removes the dominant parameter count in deep subspace clustering and permits deeper autoencoders. The closed-form derivation and the TensorFlow implementation (Listing 1) are clear and reproducible. However, the experimental evidence is confounded by architecture and training differences, and the paper provides no convergence analysis of the alternating scheme. If the mechanism were properly isolated, the method could be a useful addition, but in its current form the performance claims are not supported by the evidence presented.","major_comments":[{"comment":"The alternating scheme lacks convergence guarantees. At each iteration, B is the exact minimizer of Eq. (4) for the current latent Z, but the gradient through B is stopped and the autoencoder is updated only on the reconstruction loss (Eq. 1). This is neither coordinate descent on a single objective nor a stationary-point method for any joint loss involving self-expressiveness; a fixed point of the alternation does not necessarily satisfy any optimality condition related to subspace clustering. The statement in Section 4.1 that 'convergence is experimentally guaranteed' is not a substitute for such an analysis and does not characterize what the method converges to.","section":"Section 3.3, Algorithm 2"},{"comment":"The COIL-100 state-of-the-art claim is not supported by the experimental design. DCFSC uses a five-layer encoder/decoder trained from scratch for 175 epochs with λ=10, while DSC uses a one-layer architecture with pre-trained weights; the comparison changes architecture, pre-training, and hyperparameters simultaneously. The 27.3% error rate therefore cannot be attributed to the closed-form self-expressive layer. This is especially concerning because the small-N experiments in Section 4.1, which hold the architecture fixed, show that DCFSC is consistently worse than DSC (mean 6.13% vs 2.67% on Extended Yale B).","section":"Section 4.2, Table 4"},{"comment":"The self-expressiveness of B on the training set is tautological: B is constructed so that Z≈BZ for the final latent features, regardless of whether those features lie on a union of subspaces. In the N≫d regime of COIL-100, the linear system Z=BZ with diag(B)=0 is highly underdetermined, and the particular Frobenius-regularized solution is not necessarily block-diagonal. Since the autoencoder update never explicitly rewards union-of-subspaces structure, the paper does not establish that the closed-form layer is the operative mechanism behind the reported clustering accuracy; the result could be driven by the deep features alone.","section":"Section 3.2, Eqs. (5)-(6)"},{"comment":"The paper lacks a control experiment that isolates the effect of the closed-form self-expressive layer. For example, replacing B in Algorithm 2 with the identity matrix, or with a fixed random zero-diagonal matrix, while keeping the same deep architecture, would show whether the self-expression computation contributes to the clustering performance. Without such an ablation, it is unclear whether DCFSC is a subspace clustering method or simply an autoencoder followed by spectral clustering on a data-dependent kernel.","section":"Section 4.2"}],"minor_comments":[{"comment":"The captions say 'mean of ten trails' (typo for 'trials') and do not report any variance; please provide standard deviations or confidence intervals, since the differences discussed (e.g., ORL 14.8% vs 14.0%, COIL-100 27.3% vs 26.6%) may be within trial-to-trial noise.","section":"Figures 1 and 2"},{"comment":"The statement that DCFSC's mean error 'was significantly worse' than DSC's is not supported by any statistical test; please either provide such tests or soften the wording.","section":"Section 4.1"},{"comment":"The claim of a 'state-of-the-art result' is inconsistent with the body's acknowledgment that the S3C method in [37] reports 26.6% on COIL-100, which is better than DCFSC's 27.3%; the wording should be calibrated accordingly.","section":"Abstract and Section 4.2"},{"comment":"Equation (5) is presented without derivation; the text refers to Section 3.1 of [30] but does not reproduce the key steps. Reproducing the Lagrange-multiplier derivation would make the paper more self-contained and help readers understand the role of the diag(P) normalization.","section":"Section 3.2"},{"comment":"It should be clarified that B is still an N×N matrix that must be materialized to compute the forward pass and the final affinity matrix; the memory benefit is in trainable parameters and optimizer state, not in removing the O(N^2) object entirely.","section":"Section 3.3"}],"recommendation":"major_revision","confidential_remarks":"The manuscript reads more like a technical report than a fully peer-reviewed study. The strongest contribution is the memory-efficient closed-form self-expressive layer, but the experimental validation is too weak to support the 'state-of-the-art' claim. I would encourage the editor to ask for a revised version that adds the proposed ablation, reports error bars, and either provides a convergence analysis or reframes the paper as an empirical, memory-focused contribution. The authors should also position their work more carefully against the 26.6% result of [37] on COIL-100."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know about this one because it is a simple, honest attempt to remove the big N×N trainable matrix from deep subspace clustering. The idea: compute a closed-form self-expression matrix from the current latent features at each iteration, stop gradients, and train the autoencoder only on reconstruction. The closed-form solution itself is not new—it is the EDSC/EASER ridge regression solution—but putting it as a stop-gradient layer inside a deep network is a genuine variant. The paper derives it cleanly and the TensorFlow snippet is enough to reimplement it in an afternoon.\n\nWhat it does well: the math is mostly right (Equation 6 is the correct form; Equation 5 has a row/column scaling typo), the memory argument is plausible in spirit, and the small-N experiments at least show the method converges and beats the non-deep baselines, just not DSC. The idea of using the memory savings to train a deeper encoder on COIL-100 is attractive.\n\nWhere it is soft. First and most importantly, the closed-form solution is already the core of EDSC [11], which the paper cites but never credits for the formula. The \"very first attempt\" claim is wrong and should be fixed. Second, the COIL-100 comparison is not controlled. With N=7200 and the five-layer encoder producing about 2,592-dim features, N > d, so the equation BZ=Z becomes underdetermined per row, and the closed-form B is just the minimal-norm solution among many—it does not have to carry any block-diagonal subspace structure. Comparing the deep DCFSC against the shallow DSC does not isolate whether the closed-form layer or the depth is responsible for the error drop. A same-architecture ablation would be needed. Third, no error bars are reported, even though means of ten trials are given, and the small-N numbers are consistently worse than DSC. The abstract's \"new reference model\" claim is not supported. Finally, inverting an N×N matrix every iteration is computationally heavy; the paper does not discuss runtime or the memory of P and B beyond the parameter count.\n\nAll that said, the paper is not a waste of time. The stop-gradient closed-form layer is an interesting baseline that people in the subfield should know about. It deserves a serious referee, but I would ask for an EDSC attribution, a controlled large-N experiment, and error bars before publication. For my own work, I would be more likely to cite EDSC or EASER than this paper, but I would read a revised version.","headline":"A simple idea with a serious citation gap and an uncontrolled large-N experiment; the closed-form layer is worth knowing about, but the paper overclaims.","tokens_in":12443,"tokens_out":9646,"would_cite":false,"duration_ms":93642,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A closed-form matrix can replace the trainable self-expressive layer in deep subspace clustering.","keywords":["subspace clustering","self-expressiveness","closed-form solution","deep autoencoder","memory efficiency","spectral clustering","unsupervised learning","large-scale clustering"],"falsifier":"Track the self-expression residual $\\|Y - BY\\|_F$ during training on a dataset with strong distribution shift between encoder updates. If the residual grows while reconstruction loss falls, the closed-form layer is being computed against stale features and the method's premise fails. More directly, compare final clustering error against a version that backpropagates through the closed-form $B$ or jointly optimizes it; if backpropagation changes results materially, the stop-gradient simplification is not neutral.","tokens_in":11525,"feed_emoji":"🧮","tokens_out":6326,"duration_ms":59802,"temperature":0.7,"pith_summary":"The paper proposes a deep subspace clustering model, DCFSC, whose self-expressive layer has zero learnable parameters: each training step computes the self-representation matrix directly from the current latent features in closed form instead of optimizing an $N\\times N$ parameter matrix. The authors aim to remove the memory bottleneck that keeps deep subspace clustering stuck with shallow autoencoders on large datasets. They report that on COIL-100 this lets them train a five-layer encoder-decoder from scratch and reach a 27.3% clustering error, below the earlier deep subspace clustering baseline, while using only 0.16% of its parameters. A sympathetic reader cares because the result suggests that expensive trainable self-expression may be replaceable by a data-derived closed-form layer, opening larger datasets to deep subspace clustering.","feed_headline":"Closed-form formula replaces subspace clustering's biggest layer","feed_subtitle":"DCFSC deletes the N-by-N trainable layer, then beats all compared methods on COIL-100.","key_machinery":"The load-bearing object is the closed-form self-expression matrix $B$ derived from the solution to $\\min_{\\Theta}\\,\\|X'-\\Theta X'\\|_F^2 + \\lambda\\|\\Theta\\|_F^2$ subject to $\\mathrm{diag}(\\Theta)=0$. The formula $B_{ij}=-P_{ij}/P_{ii}$ with $P=(X'X'^T+\\lambda I)^{-1}$ turns the former trainable layer into a differentiable but stop-gradient function of the latent features, so the autoencoder can be made deeper without storing or optimizing an $N\\times N$ matrix. This is what carries the argument: it converts memory cost into a per-iteration matrix inversion and removes the joint optimization of the DSC objective.","core_discovery":"On its own terms, the paper claims that the self-expressive layer of deep subspace clustering can be replaced by the closed-form minimizer of the self-expression objective. Given latent features $X'$, the matrix $B$ is $I - P\\,\\mathrm{diagMat}(\\mathbf{1}\\oslash \\mathrm{diag}(P))$ with $P=(X'X'^T+\\lambda I)^{-1}$, i.e. off-diagonal entries $B_{ij} = -P_{ij}/P_{ii}$ and zero diagonal. During training the autoencoder is updated only by the reconstruction loss $\\|X - \\mathrm{Dec}(BX')\\|_F^2$ with gradients stopped at $B$; the affinity matrix for spectral clustering is built from the final $B$. The paper's experimental claim is that this zero-parameter self-expressive layer yields face-clustering errors on ORL close to DSC and somewhat higher errors on Extended Yale B, yet is the only deep method tested to reach 27.3% error on COIL-100 with a deeper architecture, beating all compared baselines including DSC.","pith_inferences":["If the stop-gradient alternating update converges, it behaves like a coordinate-descent analogue: the encoder is trained only to make latent features reconstruct well, while the self-expression matrix is re-fit exactly at each step. A testable prediction is that the quality of the final clustering tracks the stability of the latent features across iterations, not just the reconstruction loss.","The closed-form $B$ requires a dense $N\\times N$ matrix inversion and storage, so the model's memory savings are about optimizer state and trainable parameters, not about the affinity matrix. The next bottleneck for scaling beyond $N\\approx 10^4$ is likely the cost of forming and inverting the Gram matrix and of spectral clustering, not the autoencoder.","A direct way to separate the contribution of the closed-form layer from that of the deeper architecture would be to train DSC or a parameter-reduced variant on the same five-layer architecture; if the accuracy gap persists, the closed-form layer itself, rather than depth, is what helps on COIL-100."],"forward_implications":["On the large-scale COIL-100 benchmark, the method reaches 27.3% clustering error with a five-layer autoencoder trained from scratch, beating the compared methods including DSC, while using only 0.16% of DSC's parameters.","On the small face benchmarks under DSC's own settings, DCFSC uses 0.25% and 0.44% of DSC's parameters and ends at 6.13% versus 2.67% mean error on Extended Yale B and 14.8% versus 14.0% on ORL, still beating all non-deep baselines.","Because the self-expressive term has no parameters and no gradient, the model can train a substantially deeper autoencoder at modest memory cost: the COIL-100 model used 10.8 GB of GPU memory versus 8.6 GB for the shallow DSC model.","The absence of a trainable self-expressive layer makes DCFSC a drop-in replacement for the self-expressive component in other deep subspace clustering variants, so it can likely be combined with adversarial, self-supervised, or distribution-preserving extensions."],"supporting_citations":[{"why":"Supplies the Lagrange-multiplier closed-form solution for the self-expression objective that DCFSC adopts.","marker":"[30]"},{"why":"Defines the deep subspace clustering network, its self-expressive layer, its objective, and the benchmark settings used for comparison.","marker":"[12]"},{"why":"Establishes the self-expressiveness property and provides the SSC baseline and the affinity construction used in evaluation.","marker":"[7]"},{"why":"Shows the block-diagonal structure of the self-representation matrix and explains why the L2 norm makes the diagonal constraint tractable.","marker":"[11]"},{"why":"Provides the spectral clustering step that turns the self-expression affinity matrix into final cluster labels.","marker":"[18]"},{"why":"Supplies the COIL-100 dataset used for the large-N experiment and its standard preprocessing.","marker":"[17]"}],"fun_headline_variants":["No parameters needed: closed-form self-expression for subspace clustering","Deep subspace clustering with zero-parameter self-expressive layer","Closed-form subspace clustering: beats DSC on COIL-100","DCFSC: drop the trainable layer, keep the clustering"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method's load-bearing premise is that alternately recomputing $B$ as the exact closed-form solution for the current latent features and then updating the autoencoder only by reconstruction loss keeps $B$ a valid self-expression as the encoder changes; if the encoder drifts too fast, the final $B$ can lose the block structure needed for good clustering.","fun_headline_variants_meta":{"raw":{"variants":["No parameters needed: closed-form self-expression for subspace clustering","Deep subspace clustering with zero-parameter self-expressive layer","Closed-form subspace clustering: beats DSC on COIL-100","DCFSC: drop the trainable layer, keep the clustering"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000687,"raw_usage":{"total_tokens":3091,"prompt_tokens":895,"completion_tokens":2196,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":511,"completion_tokens_details":{"reasoning_tokens":2126}},"tokens_in":511,"tokens_out":2196,"duration_ms":15514,"temperature":1.0,"reasoning_tokens":2126,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T11:12:25.145939+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Track the self-expression residual $\\|Y - BY\\|_F$ during training on a dataset with strong distribution shift between encoder updates. If the residual grows while reconstruction loss falls, the closed-form layer is being computed against stale features and the method's premise fails. More directly, compare final clustering error against a version that backpropagates through the closed-form $B$ or jointly optimizes it; if backpropagation changes results materially, the stop-gradient simplification is not neutral.","supporting_citations":[{"cited_title":"Embarrassingly shallow autoencoders for sparse data","cited_arxiv_id":null,"evidence_quote":"Supplies the Lagrange-multiplier closed-form solution for the self-expression objective that DCFSC adopts."},{"cited_title":"Deep subspace clustering networks","cited_arxiv_id":null,"evidence_quote":"Defines the deep subspace clustering network, its self-expressive layer, its objective, and the benchmark settings used for comparison."},{"cited_title":"Sparse subspace cluster- ing: Algorithm, theory, and applications","cited_arxiv_id":null,"evidence_quote":"Establishes the self-expressiveness property and provides the SSC baseline and the affinity construction used in evaluation."},{"cited_title":"Efficient dense subspace clustering","cited_arxiv_id":null,"evidence_quote":"Shows the block-diagonal structure of the self-representation matrix and explains why the L2 norm makes the diagonal constraint tractable."},{"cited_title":"On spectral clustering: Analysis and an algorithm","cited_arxiv_id":null,"evidence_quote":"Provides the spectral clustering step that turns the self-expression affinity matrix into final cluster labels."},{"cited_title":"Columbia object image library (coil-100), 1996","cited_arxiv_id":null,"evidence_quote":"Supplies the COIL-100 dataset used for the large-N experiment and its standard preprocessing."}],"review_version":1}