{"id":"a636b865-db25-45f7-9582-cae82d1b42c9","arxiv_id":"2601.21683","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"With orthonormal linear networks and optimized layer-wise projections, local-SSL updates equal global BP-SSL updates; adding top-down and 2D-spatial structure to CLAPP then nearly matches BP-SSL on CIFAR-10, STL-10, and Tiny ImageNet.","lead":"This paper proves that some backpropagation-free local learning rules can, under idealized conditions, make exactly the same weight updates as global self-supervised backpropagation, and then uses that insight to design local rules that reach global-baseline accuracy on image benchmarks. It matters because it narrows the gap between biologically plausible local learning and standard deep learning.","discovery_kind":"unification","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Theorem 3.1 is proved pointwise under conditions that gradient training destroys: B^l is not at its optimum in the experiments, and orthonormality of W^l is not preserved by the updates. The exact-equivalence claim therefore does not apply to the trajectories that produce Table 2.","rationale":"The paper's central theoretical claim, Theorem 3.1, is internally correct as a static gradient identity: under exact orthonormality of equal-width weights and with B^l at its unique convex minimizer, the local and global gradients coincide. The proof's algebra checks out. The weakness is not the theorem but its reach. In the experiments that support the headline, both conditions are violated: B^l is updated by gradient descent rather than instantaneously optimized, and W^l is not re-orthonormalized after updates. The paper acknowledges that B^l is not at B^l_* in realistic settings and even shows degradation in Figures 2 and 3B when the ideal conditions are removed. Yet the abstract and Section 1 state that local-SSL 'implements exactly the same weight update' as global BP-SSL, which reads as a claim about training dynamics, not merely about a specially constructed point. The numerical verification in Figure 2 only measures gradients at a fixed initialization, so it does not demonstrate equivalence along a trajectory. This is load-bearing because the theoretical equivalence is the paper's main conceptual bridge between local and global learning; if it only holds at measure-zero points that the training dynamics immediately leave, it cannot explain the empirical success of CLAPP++. The empirical section may still stand on its own, but the theoretical contribution needs to be worded as a pointwise characterization under non-invariant idealizations. Since the reader's verdict was already CONDITIONAL and flagged B-optimality as the weakest assumption, my read does not change the verdict; it sharpens the reason: the theorem's conditions are not merely unrealistic in degree, they are not preserved by the very update rule the theorem prescribes.","tokens_in":23351,"tokens_out":12445,"duration_ms":126285,"concrete_test":"Run the Figure 2 setup (6-layer linear network, 128 units/layer, orthonormal W, MNIST, batch of 32) for 20 epochs of actual training: update W^l with the local gradient, update B^l with the paper's Adam rule (lr=2e-4) rather than LBFGS to the minimum, and after each epoch measure cosine similarity between the local and BP-SSL updates on held-out batches. If the similarity starts at 1.0 and drops below 1.0 immediately after the first W update, the equality is not invariant under the dynamics. As a control, re-orthonormalize W^l after each step and re-optimize B^l; if equality then persists, the two conditions (B optimality, orthonormality) are exactly what the theorem requires and are not maintained by CLAPP.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing step is not the algebra of Theorem 3.1 but the transfer of that algebra to the algorithm that actually runs. The theorem assumes (i) B^l_* = argmin_B L^l at the moment of the update ('always at the optimal value' via fast timescale, §3.1) and (ii) W^l exactly orthonormal. In the experiments behind Table 2, B^l is trained by Adam gradient steps (Appendix B), so (i) fails; the paper's own Fig. 2 and Fig. 3B show that replacing B^l_* by a non-optimal trained B^l drops the cosine similarity substantially. Moreover, (ii) is not invariant: the local update ΔW^l derived from Theorem 3.1 is a full-rank gradient step, and applying it to an orthonormal W^l leaves W^l non-orthonormal, so equality (2) cannot hold at the next step. The numerical verification in Fig. 2 computes gradients at a fixed initialization and never trains W^l, so it confirms a pointwise identity, not an equivalence of learning algorithms. Consequently the theorem does not, by itself, support the abstract's claim that local-SSL 'implements exactly the same weight update' as global BP-SSL along a learning trajectory.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper asks whether local, backpropagation-free self-supervised learning can match end-to-end backprop-based self-supervised learning. It first proves that for deep linear networks with orthonormal weight matrices and an optimally trained projection matrix B^l, the per-layer local SSL gradient equals the global BP-SSL gradient (Theorem 3.1). A second result (Theorem 3.3) shows that when layer widths shrink, using the top-layer representation as the context vector makes local updates closer to BP in a deep linear network with linear f. Guided by these observations, the paper introduces CLAPP++, CLAPP++DFB, and CLAPP++both, which add 2D spatial dependence and top-down feedback to the CLAPP objective, and reports results on CIFAR-10, STL-10, and Tiny ImageNet, including a new state of the art among local-SSL rules.","tokens_in":23645,"tokens_out":12710,"duration_ms":128058,"significance":"Theorems 3.1 and 3.3 are valuable because they give precise, if narrow, conditions under which local Hebbian-like rules reproduce BP-SSL updates exactly or approximately. The proof of Theorem 3.1 is clean and the paper is transparent about its assumptions (equal-width orthonormal layers, instantaneous optimality of B^l, normalization-free losses). The empirical study is broad, with two ablations (spatial dependence, direct feedback), multiple baselines, and detailed training details that would support replication. The main weakness is that the link between the idealized theory and the convolutional-network experiments is heuristic: the 2D spatial sharing pattern is chosen by hand (Table A2) and is not derived from the theorem. The paper would be strengthened by releasing code and by a more guarded wording of the 'matches BP-SSL' claim, which fails on STL-10.","major_comments":[{"comment":"The claim that the best local-SSL rule 'matches the performance of a comparable global BP-SSL with InfoNCE or CPC-like loss functions' is not supported on STL-10. In Table 2, CLAPP++both achieves 79.62±0.28 while BP-InfoNCE achieves 81.97±1.23, a 2.35-point gap; BP-CLAPP++ achieves 80.36±0.26, whose 95% interval does not overlap with that of CLAPP++both. The statement is defensible for CIFAR-10 and Tiny ImageNet but not for STL-10. Please revise the abstract and conclusions to report the per-dataset comparison and to use 'approaches' or 'is comparable on some datasets' instead of 'matches'.","section":"Abstract and Table 2"},{"comment":"Eq. (2) is proven as a pointwise identity at an orthonormal weight configuration where B^l is assumed to be at its instantaneous optimum. It is not a statement about the learning trajectory: the local update ΔW^l is a generic full-rank gradient step, so applying it to an orthonormal W^l leaves W^l non-orthonormal, and the theorem's premise fails at the next update. Figure 2 computes similarities at a fixed initialization and never trains W^l, so it does not demonstrate trajectory equivalence. If the abstract or contribution list is intended to claim that local-SSL 'implements exactly the same weight update' as global BP-SSL during training, the authors need to prove that orthonormality (and B-optimality) is preserved or restored along the dynamics, or explicitly state that the equality is instantaneous. This caveat should also appear in the abstract.","section":"§3.1, Theorem 3.1 and Eq. (2)"},{"comment":"The convolutional variants are described as 'theory-guided', but the theory in §3.1–3.2 does not derive the 2D spatial sharing pattern of B^l; the patch sizes n×n in Table A2 are hand-picked per dataset and layer. The only support for spatial dependence is an intuition about BP error propagation and a simulation on a 4-layer linear convnet with kernel/stride 2 (Fig. 4B). This is a heuristic, not a theorem. The paper should explicitly label the patch sizes as a hyperparameter choice and provide a sensitivity analysis over n, since the main empirical improvement comes from this choice. Otherwise the statement 'theory indicates that feedback weights should have a structured 2D spatial dependence' (contribution bullet) goes beyond what is proven.","section":"§3.3, §4 and Table A2"},{"comment":"The theorem is stated with 'B^l_* = I (e.g., forward-forward)', but the paragraph immediately after the theorem says the result has not been extended to networks with layer-activity normalization, which Forward-Forward requires according to Table 1. Because normalization is a nonlinear operation, a Forward-Forward network is not a deep linear network in the sense of Theorem 3.1. The parenthetical example is therefore misleading and should be removed or changed to a normalization-free case; the theorem's proof only covers the linear, normalization-free setting.","section":"§3.1, Theorem 3.1 statement"}],"minor_comments":[{"comment":"The displayed equation for L* in the proof of Theorem 3.3 appears to have a typo: it reads 'min_{D1} (∆z^l)^T V∅ D1 c^L', but the preceding derivation of L^{l,*,B1}=L* uses V_l D1 V_l^T c^l. If the printed equation is literal, it is not equal to L*. The intended proof is recoverable, but the typo should be fixed.","section":"Appendix A.2"},{"comment":"The theorem assumes f is convex with f' ≤ 0, but the CLAPP loss uses f(x)=max(0,1−x), which is non-differentiable at the kink. The proof uses f'. Please state whether the result holds for subgradients or restrict the theorem to differentiable convex f (the simulations use softplus, which is smooth).","section":"§3.1, Eq. (1)"},{"comment":"The figure caption and the 'Simulation for Figure 4 C' paragraph say that spatial dependence is removed in the last layer for CLAPP++ and CLAPP++DFB, but Table A2 lists patch sizes for all six layers (including L6). Please clarify how the last-layer sharing is set in this control and why it does not change classification accuracy.","section":"Fig. 4C / Appendix B"}],"recommendation":"major_revision","confidential_remarks":"The paper has a solid, correctly stated linear-network theorem and a useful empirical study, but the abstract overstates the STL-10 result and the bridge from the pointwise theorem to the convnet algorithm is heuristic. Both issues are fixable with rewording plus an explicit sensitivity analysis, so I would not reject. The authors should also be asked to state in the abstract that the exact-equivalence result holds at a fixed orthonormal configuration, not along a trained trajectory."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear colleague,\n\nThe paper is worth reading. It proves a clean equivalence result and backs it with solid benchmark gains. But the headline claim runs a bit ahead of the evidence: the exact gradient equality is a pointwise identity under idealizations that do not survive the training trajectories used in the experiments.\n\nWhat is actually new: Theorem 3.1 shows that in a deep linear network with orthonormal equal-width weights, if each layer's projection B^l is at its optimal value, then the local-SSL gradient equals the global BP-SSL gradient. The proof is elementary linear algebra, but it is correct, and the unifying notation for CLAPP/FF/SCFF is handy. The direct-feedback (DFB) variant and the 2D spatial dependence for convnets are genuinely new algorithmic ideas, and they are motivated by the theory. The empirical results are real: CLAPP++ and its DFB/both variants outperform previous local-SSL methods on CIFAR-10, STL-10, and Tiny ImageNet, and come close to (or beat) the BP baselines on CIFAR and Tiny. That is a meaningful step for the subfield.\n\nThe soft spots are proportionate. The main one: Theorem 3.1 is a fixed-point statement. It assumes B^l is at its optimum and W^l is orthonormal at the moment of the update. Gradient descent destroys both: after a step, W^l is no longer orthonormal, and in the convnet experiments B^l is trained by Adam, not held at an optimum. Figure 2 verifies gradients at a fixed initialization without updating W, so it confirms a pointwise identity, not an equivalence of learning algorithms. The abstract's \"implements exactly the same weight update\" is therefore too strong if read along a learning trajectory. The paper does disclaim that the theorem doesn't carry to nonlinear networks, but the trajectory issue is left unqualified in the abstract.\n\nSecond, the \"matches ... InfoNCE\" claim is not supported on STL-10: CLAPP++both gets 79.62±0.28 vs BP-InfoNCE 81.97±1.23; the intervals don't overlap. It does match on CIFAR-10 and Tiny ImageNet.\n\nThird, practical reproducibility: no code is shipped, and the spatial patch sizes in Table A2 are hand-picked per dataset/layer with no sensitivity analysis. Minor, but a referee should ask for it.\n\nOverall: the theory is sound under its stated assumptions, the empirical contribution is real, and the authors are honest about limitations. The overclaims are fixable with wording and a trajectory-level caveat. This should go to peer review rather than being desk-rejected; I would expect a solid, not transformative, paper after revision.","headline":"Real theory, good empirical gains, but the exact-equivalence claim is pointwise under idealizations; still worth refereeing with requested revisions.","tokens_in":24191,"tokens_out":3365,"would_cite":true,"duration_ms":32102,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"In deep linear networks with orthonormal weights and optimally learned per-layer projections, local self-supervised learning produces exactly the same weight updates as global backpropagation; the theory then yields local convnet rules that","keywords":["local learning","self-supervised learning","backpropagation","deep linear networks","contrastive learning","CLAPP","direct feedback","spatial dependence"],"falsifier":"Train a deep linear network with orthonormal weights and a CLAPP-style loss, updating B^l by gradient descent on the same timescale as W^l, and measure the cosine similarity between the local gradient and the global BP-SSL gradient across layers; if similarity drops below 1 in early layers, the instantaneous optimality of B^l—not orthonormality alone—is doing the load-bearing work in Theorem 3.1.","tokens_in":23167,"feed_emoji":"🧠","tokens_out":8388,"duration_ms":80780,"temperature":0.7,"pith_summary":"The paper tries to establish that local self-supervised learning rules are not inherently weaker than global backpropagation-based self-supervised learning (BP-SSL). It proves that in deep linear networks with orthonormal weight matrices, a local layer-wise contrastive loss with an optimally learned per-layer projection B^l produces exactly the same gradient for every feedforward weight as global BP-SSL would. Starting from that theorem, the paper modifies CLAPP-style local rules in two theory-guided ways—letting B^l depend on 2D spatial location in convolutional networks, and routing the reference signal from the top layer instead of each local layer—and shows that the resulting variants match the downstream classification accuracy of a comparable global BP-SSL baseline on CIFAR-10, STL-10, and Tiny ImageNet. A sympathetic reader would care because the result reframes the local/global gap as an optimization and architecture problem rather than an algorithmic impossibility, and it gives biological learning rules a route to global objectives without a backward pass.","feed_headline":"Local learning matches self-supervised backpropagation","feed_subtitle":"A theory-guided variant of CLAPP matches end-to-end backprop on three standard image datasets.","key_machinery":"The load-bearing object is the per-layer projection matrix B^l (or the identity, in Forward-forward-style rules) that maps a reference vector c^l onto the layer's activity space inside the local loss. The theorem's key algebraic fact is B^l_* = (W^L ... W^{l+1})^T B^L_* (W^L ... W^{l+1}), an identity that orthonormal weight matrices make available and that converts the local per-layer gradient into the exact backpropagated gradient. Two structural variants carry the practical argument: direct feedback, which replaces the reference c^l with the top-layer activity z^L and improves gradient similarity when layers shrink, and 2D spatial dependence, which lets B^l vary by spatial location in conv","core_discovery":"The central claim is Theorem 3.1: in a deep linear network with orthonormal weight matrices and a layer-wise contrastive loss of the form f(z_pos^T B^l c_pos) + f(-z_neg^T B^l c_neg), the gradient of the locally minimized loss with respect to W^l equals the gradient that global backpropagation computes for the last-layer loss, whenever B^l is the identity or the unconstrained minimizer of the local loss. The proof works by identifying the optimal per-layer projection as B^l_* = (W^L ... W^{l+1})^T B^L_* (W^L ... W^{l+1}), which orthonormality turns into an exact match between local and backpropagated gradient directions. The authors then prove a companion result for shrinking-width linear ne","pith_inferences":["Restating the paper's own proof as a design principle: a local rule whose per-layer projection is trained to mimic the backpropagated gradient should inherit BP-SSL's behavior. This predicts that richer, data-dependent families of B^l—nonlinear projections, or projections trained to directly minimize gradient mismatch—would shrink the residual gap in deep nonlinear networks further.","The patch sizes that control spatial sharing of B^l are set by hand per dataset and layer, not derived from the theory. A testable extension would learn or infer those sizes from the data; if performance survives, the mechanism is spatial structure itself, not the extra tuned parameters.","If the same spatial-dependence and direct-feedback ideas transfer to non-contrastive local losses (variance/decorrelation losses) and to larger modern architectures, the practical upshot would be that self-supervised representation learning can run on neuromorphic or energy-constrained hardware without a global backward pass.","The biological analogy the authors draw—a top-layer signal gating plasticity like apical dendritic input—could be turned into a concrete prediction: a cortex-like circuit with only a delayed global novelty/prediction signal should learn hierarchical representations as well as one with layer-specific error propagation."],"forward_implications":["In the idealized regime of Theorem 3.1, a local-SSL update is not merely a heuristic approximation of BP-SSL; it is the same gradient, so any local/global gap must come from optimization or architecture rather than from locality itself.","When the number of neurons shrinks across layers, feeding the top-layer activity back as the reference signal is provably closer to the global BP-SSL gradient (in the linear case) than using each layer's own activity, and this transfer extends numerically to ReLU networks.","In convolutional networks, giving the per-layer projection B^l a 2D spatial structure increases the cosine similarity of local and BP-SSL gradients and improves downstream accuracy; direct feedback adds a further small improvement.","The CLAPP++ variants match the accuracy of a same-architecture global BP-SSL baseline on CIFAR-10, STL-10, and Tiny ImageNet, and improve on the previous state of the art for local-SSL on those benchmarks.","CLAPP++DFB stores a single top-layer context vector instead of a context vector per layer, which lowers the memory footprint of training."],"fun_headline_variants":["Local learning matches backprop on image benchmarks","Local SSL closes gap to backprop self-supervision","Theory-guided local learning rivals backprop","Local rules equal backprop in self-supervised nets","Matching backprop: local learning finally does it"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The load-bearing premise is that the per-layer projection B^l is always at its exact optimal value when the feedforward weights are updated, because B^l is assumed to evolve on a much faster timescale; the actually trained convnets update B^l by gradient steps, so the exact equality in Theorem 3.1 is not guaranteed to hold there.","fun_headline_variants_meta":{"raw":{"variants":["Local learning matches backprop on image benchmarks","Local SSL closes gap to backprop self-supervision","Theory-guided local learning rivals backprop","Local rules equal backprop in self-supervised nets","Matching backprop: local learning finally does it"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00055,"raw_usage":{"total_tokens":2465,"prompt_tokens":750,"completion_tokens":1715,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":494,"completion_tokens_details":{"reasoning_tokens":1643}},"tokens_in":494,"tokens_out":1715,"duration_ms":15066,"temperature":1.0,"reasoning_tokens":1643,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-03T06:52:59.043687+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train a deep linear network with orthonormal weights and a CLAPP-style loss, updating B^l by gradient descent on the same timescale as W^l, and measure the cosine similarity between the local gradient and the global BP-SSL gradient across layers; if similarity drops below 1 in early layers, the instantaneous optimality of B^l—not orthonormality alone—is doing the load-bearing work in Theorem 3.1.","supporting_citations":[],"review_version":1}