{"id":"58a74a28-30ff-4c3d-96cd-293fb007f5ba","arxiv_id":"2411.17961","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"ESS-ReduNet speeds ReduNet training by dynamically boosting the expansion operator and correcting membership estimates with label-derived Bayesian posteriors, reporting more than 10x faster convergence on several datasets.","lead":"ESS-ReduNet adds two mechanisms to ReduNet, a white-box neural network built from rate-reduction math: a growing weight on the expansion operator and a label-derived Bayesian correction for membership estimates. On seven datasets it reports much faster convergence and better classifier accuracy than the original ReduNet, but without code, error bars, or a formal derivation of why the schedule works.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Speedup and accuracy gains may reflect an unmatched stopping rule: ReduNet is run to 3000 layers even after its own condition-number plateau, then compared against an early-stopped ESS-ReduNet.","rationale":"The central claim is quantitative, so the most load-bearing assumption is that ReduNet and ESS-ReduNet are compared under equivalent stopping conditions. The paper's own Figure 3d says ReduNet's condition number stabilizes around layer 1600, while Table 1 reports ReduNet's layer as '> 3000'; Section 3 acknowledges that running ReduNet past convergence degrades feature quality. Therefore the reported ReduNet accuracy of 0.65 may be an over-trained value, and the speedup ratio may be smaller than advertised. This is concrete and testable. The unproved expansion schedule and the Bayesian correction are secondary concerns: even if they are heuristics without guarantees, the empirical comparison could still be made fair. I do not see grounds to reject outright; the method may still improve ReduNet, but the headline numbers should be recomputed with a matched condition-number stopping rule before acceptance. The reader's rationale noted the upper-bound nature of the 10x claim but did not explicitly flag the over-training discrepancy between Figure 3d and Table 1, hence partial agreement.","tokens_in":16351,"tokens_out":9138,"duration_ms":86381,"concrete_test":"Re-run both methods on ESR and at least one other dataset with an identical condition-number stabilization rule: stop when the maximum relative change in κ over a 50-layer window falls below a pre-specified threshold, and report the layer count and SVM/KNN/NSC accuracy at that stopping point for both ReduNet and ESS-ReduNet. If ReduNet's stopping layer on ESR is approximately 1600 and its SVM accuracy exceeds 0.65, the '>10x' and '47%' headline claims fail; if ReduNet's condition number genuinely has not stabilized before 3000 under the same rule, the claims stand.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is quantitative: more than 10x fewer layers and a 47% SVM accuracy gain on ESR. Such claims require matched stopping conditions, but the evidence does not provide them. On ESR, the paper's own Figure 3d states ReduNet's condition number stabilizes around layer 1600, while Table 1 lists ReduNet's convergence layer as '> 3000'. The paper then evaluates ReduNet's SVM accuracy (0.65) at this over-run endpoint, even though Section 3 itself says 'further training of a poorly constructed network degrades feature quality.' ESS-ReduNet is stopped by the condition-number criterion at layer 199 and evaluated there. Thus the '>10x' speedup (3000/199 ≈ 15) and the '47% SVM improvement' (0.65 to 0.96) conflate the method's effect with the effect of stopping before degradation. If ReduNet were stopped at its own condition-number plateau, the ESR speedup would be roughly 1600/199 ≈ 8x, and its SVM accuracy at that layer could be substantially higher than 0.65. No matched comparison is reported for any dataset; the Gas row, where ESS-ReduNet's SVM is 0.97 versus ReduNet's 0.98, further indicates the fairness issue is material.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper proposes ESS-ReduNet, a modification of the white-box ReduNet architecture that aims to improve subspace separability and accelerate convergence. Two main mechanisms are introduced: a Bayesian-inference correction of the class-membership estimates used in the layer-wise updates (so that label information can be reused at test time without inconsistency), and a dynamically weighted expansion operator that is strengthened when the training appears to stagnate. A condition-number-based stopping rule is also proposed as an auxiliary convergence criterion. The paper reports experiments on seven datasets, claiming more than a 10-fold reduction in the number of layers needed for convergence on four of them and large gains in SVM/KNN/NSC accuracy of the transformed features (e.g., 0.65 to 0.96 SVM accuracy on ESR). An ablation study indicates that both the Bayesian module and the dynamic expansion module contribute to the improvement.","tokens_in":16640,"tokens_out":4503,"duration_ms":39557,"significance":"If the reported gains are robust, the paper would make ReduNet training substantially more practical and would provide a useful case study in combining white-box, layer-wise constructed networks with label-derived Bayesian corrections. The geometric interpretation of the expansion operator (Section 4.2 and Appendix A) is a coherent and useful addition, and the ablation in Figure 9 provides evidence that both proposed modules are active ingredients. However, the central quantitative claims (10x layer reduction and 47% accuracy gain) are currently supported only under a comparison protocol that does not apply matched stopping rules to the baseline, and the hand-crafted expansion schedule is not analyzed for sensitivity. The paper does not release code or report multiple seeds, which limits the reproducibility of the empirical claims.","major_comments":[{"comment":"The comparison between ESS-ReduNet and ReduNet is not made under matched stopping conditions. On the ESR dataset, Figure 3d shows ReduNet's condition number stabilizing around layer 1600, and the text in Section 3 explicitly says that further training of a poorly constructed network degrades feature quality. Yet Table 1 reports ReduNet's convergence layer as '>3000' and evaluates its SVM accuracy (0.65) at that over-run endpoint, while ESS-ReduNet is stopped at layer 199 by the condition-number criterion. This mismatched protocol inflates both the '>10x speedup' and the '47% SVM accuracy gain' claimed in the abstract. The authors should report ReduNet's accuracy at its own condition-number plateau (or at the same layer count as ESS-ReduNet) and apply the identical stopping rule to both methods across all datasets.","section":"Section 5.3, Table 1; Section 3, Figure 3d"},{"comment":"The expansion schedule w(tau)=min(exp(tau),u) with tau incremented by 0.1 per error layer introduces two free parameters (the increment and the cap u) that are fixed without sensitivity analysis. The paper asserts that this schedule robustly enlarges the spanned space to decouple class subspaces, but no ablation over these parameters is given, and no convergence guarantee is provided. To support the claim that the method is not overly sensitive to hand-tuned dynamics, the authors should report results for at least a few values of the increment and cap, or justify their chosen values with a principled derivation.","section":"Section 4.2, Eq. (11); Algorithm 1 lines 10-13"},{"comment":"All experimental results appear to be single runs with no error bars, multiple seeds, or significance testing, and no code is released. Since the paper's central claims are quantitative (layer reduction and accuracy improvement), the authors should release code and report statistics over multiple random initializations and train/test splits, or justify why single runs are sufficient for these claims.","section":"Section 5 (all experiments)"}],"minor_comments":[{"comment":"The phrase 'spannd space' appears; it should be 'spanned space'.","section":"Appendix A"},{"comment":"The word 'compresssion' should be 'compression' in the sentence about white-box transformers.","section":"Section 2.2"},{"comment":"The legend entries 'Bayes_1' through 'Bayes_9' and 'ReduNet_1' through 'ReduNet_9' are not explained in the text; the caption should state that they correspond to different channel numbers Nc.","section":"Figure 9a"},{"comment":"The subscripts in the posterior notation p_ij and P(z in C_i | z -> C_j) are not clearly defined; please specify the direction of the conditional (i.e., which index corresponds to the observed class and which to the true class) in the text.","section":"Section 4.1, Eq. (7)"},{"comment":"The stopping criterion 'no change on condition number' is not quantified; the authors should define the tolerance and the number of layers over which the change is checked, so that the stopping rule is reproducible.","section":"Section 4.3"}],"recommendation":"major_revision","confidential_remarks":"The central idea is interesting and the ablation provides some support for the method, but the current comparison protocol undermines the headline quantitative claims. The paper would be much stronger if the authors redo the experiments with matched stopping rules, add sensitivity analysis for the schedule parameters, and release code. The fit to the journal's scope is reasonable."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe two modules here are real: a time-varying weight on the expansion operator and a Bayesian posterior correction of the membership estimates, both plugged into ReduNet's forward construction. That combination is new in the ReduNet literature, and the ablation in Figure 9 gives credible evidence that both pieces contribute. The geometric interpretation of the expansion operator as a residual projection is also clearly explained. So there is substance here.\n\nThe problem is the headline numbers. The 10x speedup and the 47% SVM gain on ESR come from comparing ESS-ReduNet early-stopped at layer 199 against ReduNet run to 3000 layers, even though the paper's own Figure 3d shows ReduNet's condition number plateauing around layer 1600 and the text admits that continuing to train a poorly constructed network degrades feature quality. That is not a matched comparison. If you stop ReduNet at its own plateau, the speedup is closer to 8x and the SVM accuracy is likely higher than the reported 0.65. The Gas row, where ESS-ReduNet gets 0.97 SVM against ReduNet's 0.98, shows this matters. The stress-test note is correct.\n\nOther soft spots: no code, no seeds or error bars, and the expansion schedule w(τ)=min(exp(τ),u) is hand-set with no sensitivity analysis. The Bayesian posterior uses labels during training, which is fine because the correction is cached and reused at test time; that is not circular. The lack of convergence guarantees is not fatal for an empirical paper, but it should be stated as such.\n\nWhat the paper does well: it identifies a real failure mode of ReduNet (the vicious cycle of bad estimates and entangled subspaces), and the ablation supports the cure. The method is simple and plausibly useful. But the central quantitative claim needs to be re-run with matched stopping rules, multiple seeds, and ideally code release.\n\nMy recommendation: send it to peer review, but expect major revision. The comparison must be redone, and the authors should report ReduNet's performance at its own condition-number plateau, not at an arbitrary 3000 layers. If the gains survive that, it is a useful contribution to the representation-learning subfield. If not, the paper still has a contribution worth stating more modestly.","headline":"Promising plug-in fixes for ReduNet, but the headline speedup relies on an unfair stopping-rule comparison and needs re-benchmarking.","tokens_in":17156,"tokens_out":2098,"would_cite":false,"duration_ms":17743,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["68T07","62F15"],"pacs":[],"model":"deepseek-v4-flash","headline":"Dynamic expansion weighting plus Bayesian membership correction makes ReduNet converge in about a tenth of the layers and yields better-separated, more classifiable features.","keywords":["ReduNet","maximal coding rate reduction","subspace learning","Bayesian inference","representation learning","early stopping","condition number","membership estimation"],"falsifier":"Train ESS-ReduNet on a dataset whose intrinsic dimension equals the lifted feature dimension, leaving no orthogonal complement for the expansion operator to enlarge the span; if the estimation-function errors still vanish and convergence remains rapid, the expansion schedule is not the operative mechanism, whereas a stall would confirm the paper's causal story.","tokens_in":16146,"feed_emoji":"⚡","tokens_out":8075,"duration_ms":68067,"temperature":0.7,"pith_summary":"This paper tries to establish that ReduNet, the white-box network built from maximal coding rate reduction, can be made both faster and more accurate by fixing two sources of error: the limited span of the feature space and the unreliable membership estimates used in feature updates. The proposed ESS-ReduNet dynamically increases the weight of the expansion operator, the term that moves features into the complement of the spanned space, whenever the estimation function misclassifies samples, and it uses Bayesian inference to blend label information into the membership estimates in a way that can be reused at test time. It also adds a condition-number stability check as an early-stopping rule. If the paper is right, ReduNet becomes practical, needing roughly ten times fewer layers to converge while producing transformed features that are better separated, with SVM accuracy rising from 0.65 to 0.96 on the ESR dataset.","feed_headline":"Faster ReduNet: 10x fewer layers with dynamic expansion and Bayes","feed_subtitle":"Weighting the expansion operator and correcting membership with Bayesian posteriors lifts ESR SVM accuracy from 0.65 to 0.96.","key_machinery":"The central object is the per-layer ReduNet update $z_{\\ell+1}\\propto z_\\ell+\\eta(E_\\ell z_\\ell-\\sum_j\\gamma_j C_\\ell^j z_\\ell \\hat{\\pi}_j(z_\\ell))$, where $E_\\ell=\\alpha(I+\\alpha Z_\\ell Z_\\ell^T)^{-1}$ is the expansion operator (shown to be approximately the ridge-regression residual, i.e., projection onto the complement of the spanned space) and $C_\\ell^j$ are compression operators. ESS-ReduNet's mechanism is the weighted expansion $w(\\tau_\\ell)E_\\ell z_\\ell$ with $w(\\tau)=\\min(\\exp(\\tau),u)$ and $\\tau$ incremented by 0.1 on every layer with estimation errors, combined with Bayesian-corrected memberships $p_c^i(z_\\ell)$ defined through the confusion posterior $P(z\\in C_i\\mid z\\to C_j)$. A condition number $\\sigma_{\\max}/\\sigma_{\\min}$ of the matrices in $E_\\ell$ and $C_\\ell^j$ serves as an auxiliary early-stopping signal. Together these pieces enlarge the overall spanned space so class subspaces decouple, and correct the membership estimates so samples move toward the right subspace.","core_discovery":"ReduNet builds each layer's parameters from the features of the previous layer, updating features by a gradient step of a maximal coding rate reduction objective. Its expansion operator $E_\\ell$ pushes features into the complement of the spanned space, and its compression operators $C_\\ell^j$ pull samples toward the subspace of class $j$, using estimated memberships instead of labels. ESS-ReduNet modifies this update in two ways: it multiplies the expansion term by a truncated exponential weight $w(\\tau_\\ell)=\\min(\\exp(\\tau_\\ell),u)$ that grows whenever the estimation function misclassifies a sample, and it replaces the raw membership estimate with a Bayesian posterior $p_c^i(z)=\\sum_j P(z\\in C_i\\mid z\\to C_j)\\,\\hat{\\pi}_j(z)$ that is computed during training by comparing estimates with labels and then reused at test time. The paper reports that this joint modification reduces the number of layers needed for convergence from over 3000 to roughly 100 to 300 on four benchmark datasets, and on the ESR dataset it reduces misclassified membership estimates to zero by layer 19 while lifting SVM accuracy on the transformed features from 0.65 to 0.96.","pith_inferences":["The Bayesian posterior effectively converts training-time estimation errors into a confusion matrix, so ESS-ReduNet behaves like a self-training procedure that injects label information only when the unsupervised estimates are unreliable; the same correction could apply to other iterative subspace-clustering or self-supervised methods with soft memberships.","The fixed schedule $\\tau_{\\ell+1}=\\tau_\\ell+0.1$ is a single global knob; on datasets with more classes or different scales, the same increment may overshoot or undershoot, so a dataset-dependent schedule driven by the observed misclassification rate is a natural testable extension.","The geometric reading of the expansion operator as a residual projection suggests a convergence proof could be attempted under a margin condition on the class subspaces; the paper does not provide one, leaving a formal guarantee open.","Because the condition number stabilizes earlier than the objective function, this stopping heuristic could serve as a general diagnostic for layer-wise-constructed white-box networks beyond ReduNet."],"forward_implications":["ReduNet training becomes practical: instead of thousands of layers, a few hundred suffice for convergence on standard benchmarks, cutting the compute and storage of parameters.","The plug-in modules, weighted expansion and Bayesian correction, carry over to the Fourier version of ReduNet and improve its accuracy as well.","The condition-number halting rule gives a cheap stopping signal that prevents the feature-quality degradation seen when a poorly constructed network trains too long.","Downstream classifiers such as SVM, KNN, and NSC benefit from the more orthogonalized features, with large gains on datasets that were previously challenging, including the ESR SVM accuracy jump from 0.65 to 0.96."],"supporting_citations":[{"why":"Defines ReduNet and the MCR2 objective that this paper modifies, providing the expansion and compression operator construction.","marker":"[Chan et al., 2022]"},{"why":"Supplies the coding-rate measure that underlies the MCR2 objective and the notion of subspace compression.","marker":"[Ma et al., 2007]"},{"why":"Justifies the geometric reading of the expansion operator as a residual projection onto the complement of the spanned space.","marker":"[Strang, 2012]"},{"why":"Provides the ridge-regression solution and Woodbury identity used to connect the expansion operator to residual projection.","marker":"[van Wieringen, 2023]"},{"why":"Supplies the condition-number criterion for ridge regression that motivates the early-stopping rule.","marker":"[Tabeart et al., 2019]"},{"why":"Establishes the SVM, KNN, and NSC evaluation protocol for features transformed by rate-reduction networks.","marker":"[Yu et al., 2020]"},{"why":"Provides the Kaggle version of the ESR dataset used in the case study and accuracy comparison.","marker":"[Ur-Rashid, 2018]"}],"fun_headline_variants":["ESS-ReduNet: 10x faster convergence with Bayes-corrected subspaces","Dynamic expansion + Bayesian inference: ReduNet 10x speedup","Bayes membership fixes ReduNet, cutting layers by 10x","Weighted expansion and Bayes: ReduNet converges 10x faster","ESS-ReduNet: 10x fewer layers, SVM accuracy 0.96"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The claimed speedup rests on the unproved assumption that the hand-tuned rule for strengthening the expansion term, increasing its weight by a fixed amount every time any sample is misclassified, reliably separates class subspaces and never steers feature updates in the wrong direction, on any dataset.","fun_headline_variants_meta":{"raw":{"variants":["ESS-ReduNet: 10x faster convergence with Bayes-corrected subspaces","Dynamic expansion + Bayesian inference: ReduNet 10x speedup","Bayes membership fixes ReduNet, cutting layers by 10x","Weighted expansion and Bayes: ReduNet converges 10x faster","ESS-ReduNet: 10x fewer layers, SVM accuracy 0.96"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000834,"raw_usage":{"total_tokens":3691,"prompt_tokens":1052,"completion_tokens":2639,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":668,"completion_tokens_details":{"reasoning_tokens":2537}},"tokens_in":668,"tokens_out":2639,"duration_ms":18430,"temperature":1.0,"reasoning_tokens":2537,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T11:39:38.626116+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train ESS-ReduNet on a dataset whose intrinsic dimension equals the lifted feature dimension, leaving no orthogonal complement for the expansion operator to enlarge the span; if the estimation-function errors still vanish and convergence remains rapid, the expansion schedule is not the operative mechanism, whereas a stall would confirm the paper's causal story.","supporting_citations":[{"cited_title":"Re- duNet: A white-box deep network from the principle of maximizing rate reduction","cited_arxiv_id":null,"evidence_quote":"Defines ReduNet and the MCR2 objective that this paper modifies, providing the expansion and compression operator construction."},{"cited_title":"Linear algebra and its appli- cations","cited_arxiv_id":null,"evidence_quote":"Justifies the geometric reading of the expansion operator as a residual projection onto the complement of the spanned space."},{"cited_title":"van Wieringen","cited_arxiv_id":null,"evidence_quote":"Provides the ridge-regression solution and Woodbury identity used to connect the expansion operator to residual projection."},{"cited_title":"Tabeart, Sarah L","cited_arxiv_id":null,"evidence_quote":"Supplies the condition-number criterion for ridge regression that motivates the early-stopping rule."}],"review_version":1}