Pith.

Machine Learning · arXiv:2611.17019 v1 · submitted 12 Nov 2026 · 33 references

Temporal Regularized Learning: Self-Supervised Learning Local in Space and Time

Davide Wiest

The paper says

“Layer-local self-supervised objectives remove the backward pass between layers, but almost all still compute their loss over a batch of samples and are therefore local in space but not in time.”

more of the abstract

We introduce Temporal Regularized Learning (TeReL), a spatially and temporally local learning rule for self-supervised representation learning from sequential input streams. The key idea is to instantiate a VICReg-style objective with temporally adjacent positives and non-differentiated running population statistics, yielding a per-neuron online loss with bounded state. The objective is a soft form of Slow Feature Analysis, and its gradient is a Hebbian update of the feedforward weights and an anti-Hebbian update of one lateral matrix per layer. The aim of the experiments is to show that the rule works in a small network, not to compete with backpropagation. On a class-chunked MNIST stream, a two-layer fully connected network trained with batch size one outperforms random features (96.48% versus 95.09% linear-probe accuracy) and classical SFA, and a variant that relaxes both locality constraints is comparable to supervised backpropagation. Because training and inference use the same computation, the network can keep learning from unlabeled data during inference without loss of accuracy.

from the abstract

Pith says

Machine review · deepseek-v4-flash · 13 Nov 2026 · what was read was not recorded · not checked by a person

It can be wrong. Challenge this review or reply to it.

Holds up, with conditions. A clean, honest, reproducible local learning rule with real but small gains — the sharpest unresolved question is the boundary oracle the class-chunked stream hands the network. This line is wrong

2 major objections · 4 minor

How often objections from reviews like this one are right: not yet measured

Are you an author? What you can do.

Not checked: figures, code, what the 33 references say.

The paper says, at Section 4, Eq. (2), Section 2.4

The review did not quote this passage.

Pith says: Objection 1 of 2 · major

The empirical case for the temporal signal is confounded by the boundary flag I_t. In the described stream, chunks are class-homogeneous and sequence boundaries coincide with class changes, so I_t is a per-sample oracle that tells the layer when the latent class has changed: within a chunk it enforces z_t close to p_t, and at the first sample of a new class it suppresses that pull.

more of objection 1

The two ablations in Figure 2b do not separate these effects: shuffling changes both the same-label adjacency and the alignment of boundaries with label changes, while removing the temporal term removes both the within-chunk pull and the boundary gating. The limitation is acknowledged in Section 6 (sentences b93 and b133), but the specific boundary-oracle problem is not. Please add an ablation that fixes boundary positions while destroying class homogeneity, an ablation that keeps class chunks but removes the boundary gating (I_t = 1 across boundaries), and ideally a truly continuous ordering, so that the contribution to H2 can be attributed to slow-feature learning rather than to the experimenter-provided segmentation.

The paper says, at Section 2.3, Eqs. (5)-(6)

The review did not quote this passage.

Pith says: Objection 2 of 2 · major

The update used in the experiments is not exactly the gradient derived in Proposition 1: with the selected kappa = 0.575, s_t = b_t - eta kappa M_t b_t is a heuristic modification, and Eq. (6) is only the exact feedforward gradient when kappa = 0.

more of objection 2

No statement or proof is given that this one-step correction preserves the descent direction of the VICReg-style loss. Moreover, the 'no covariance term' ablation sets lambda_C = 0 but leaves M_t active in s_t and in Eq. (7), so it does not isolate the covariance loss from the lateral-correction mechanism. Please report the kappa = 0 and lambda_C = 0 conditions separately, and either prove the correction's descent property or mark the correction explicitly as an empirical design choice outside the derived objective.

4 minor objections

The paper says, at Appendix A, Table 1

Pith says: minor objection 1 of 4

The compute budgets differ across methods: TeReL-Offline consumes 1,000,000 samples while supervised offline consumes 500,000 samples. The paper states that the comparison is about representation quality rather than equal compute, but a sentence reporting a matched-sample or matched-epoch condition would make the 'comparable to supervised backpropagation' statement less open to misinterpretation.

The paper says, at Section 2.1 and Section 2.3

Pith says: minor objection 2 of 4

The phrase 'per-neuron online loss' is slightly overstated because the covariance term and the lateral correction couple neurons through M_t; 'neuron-local loss with same-layer lateral coupling' would be more precise.

show 2 more

The paper says, at Appendix A

Pith says: minor objection 3 of 4

The selected hyperparameters are listed, but the search ranges and the validation criterion used for model selection are not. Reporting them would improve reproducibility.

The paper says, at References

Pith says: minor objection 4 of 4

Reference [27] is hosted on the author's own page rather than in an archival venue; if it is used as a prior-method citation, a versioned DOI or an archival source would be helpful.

What would settle it

Run the main TeReL protocol on the same class-chunked MNIST stream under two boundary manipulations: (i) no boundaries, I_t = 1 for every observation so the temporal term always pulls toward the previous sample; (ii) the same number of boundaries but placed at random positions, uncorrelated with label changes. Compare fixed-encoder linear-probe accuracy against the reported TeReL 96.48%, shuffled-order 95.40%, and no-temporal-term 94.98% values. If either condition drops to the shuffled or no-temporal level, the central gain depends on the label-aligned boundary oracle rather than on temporal coherence alone.

from the skeptic's reading

How the review went

4 readings by deepseek-v4-flash · 13 Nov 2026

  1. Reader holds up, with conditions confidence moderate
  2. Skeptic partly agrees 1 min 21 s · 12,766 tokens read, 11,273 written
  3. Editor wrote the verdict above 24 s · 14,332 tokens read, 851 written
  4. Referee major revision: 2 major objections, 4 minor 1 min 14 s · 13,136 tokens read, 10,863 written

Not run on this paper: the simulated authors' defence.

Pith did not check this review's objections against the paper.

The review in full

The review in brief, machine-written

Worth reading if you work on local or neuromorphic learning rules. TeReL is a VICReg-style objective made per-neuron and temporally local: adjacent positives, detached running statistics, and a lagged lateral matrix yield a Hebbian feedforward / anti-Hebbian lateral update. That combination is genuinely new, the derivation in Proposition 1 is clean, and the paper is explicit about what is a derived gradient versus a design choice. What is solid: the protocol. Held-out test split, validation for model selection, five seeds with paired 95% intervals, raw values in the appendix, code and a Zenodo DOI. The ablations behave the way they should — the variance term collapses, the covariance term diverges, and both shuffling and dropping the temporal term degrade accuracy. That is a real sanity check and it passes. The paper also scopes itself honestly: proof of concept in a small network, not a backprop competitor. I agree with the reader's soundness-7 and low-circularity scores; the linear probe is external to the objective, and nothing fitted is dressed up as a prediction. The stress-test note holds up, and it is the sharpest version of the real problem. The It flag in Eq. (2) tells the network exactly when a class chunk ends — pull toward the previous activation or suppress that pull. Because chunk boundaries are precisely the label changes, this is an experimenter-provided boundary oracle, not something the rule discovers. Neither ablation — shuffling, or removing the temporal term — separates boundary gating from temporal adjacency, so you cannot tell how much of the 1.39-point gain over random features relies on being told when the class changes. Natural streams do not come with boundary flags. The authors acknowledge the constructed stream in Section 6, but the oracle point is sharper than the acknowledgment and deserves its own experiment. The gains themselves are modest: 1.39 points over a random encoder, and Layer-local SupCon — which uses labels — beats TeReL by half a point. The win over classical SFA is the cleaner result. The lateral correction strength and mixing weight are hand-set rather than derived from a single objective, and the online continuation result comes after validating the step scale on the validation stream; it preserves rather than improves. All of this is stated in the paper, nothing hidden. Send it to review. It is a clean, honest, reproducible small contribution with one real open question — whether the rule works without the boundary oracle and on natural streams — and a referee should push on exactly that.

The strongest objection

Section 4 constructs the stream as randomly permuted class-homogeneous chunks of 16, and Section 2.4 says that at a sequence boundary only the validity of p_t is cleared. Because the chunk boundaries are exactly the label changes, the I_t term in Eq. (2) gives the network a precise oracle for when the 'slow feature' changes: within a chunk it pulls z_t toward the previous activation, and at the first sample of a new class it suppresses that pull. The reported ablations do not separate these two signals. Shuffling the stream destroys same-label adjacency but keeps random sequence boundaries; removing the temporal term removes the whole temporal objective. Neither condition tests whether TeReL would work when the stream is continuous or when boundary times are unknown, as in natural video or sensor data. This is a sharper version of the acknowledged limitation in Section 6 (sentences b93, b133): the unresolved issue is not only that positives are label-sorted, but that the algorithm is told exactly when the latent class changes. If the gain over random features is largely due to this boundary oracle, the evidence for a self-supervised temporal learning rule is considerably weaker than the abstract suggests.

The reader's weakest assumption is that the temporal signal is injected by the label-chunked ordering. I agree, and I sharpen it: the stream also provides exact boundary times, and these times enter the loss directly through I_t in Eq. (2) and through the p_t reset in Section 2.4. The authors are transparent about the constructed order and explicitly list natural-stream evaluation as the most important extension (Section 6), so this is not a hidden flaw. It is nonetheless load-bearing for the broad 'self-supervised temporal representation learning' framing, because the current experiments cannot tell whether TeReL learns from temporal coherence or from a label-derived segmentation signal. The math in Proposition 1 is sound, the protocol is careful about validation/test separation, and the ablations support H2 within the constructed setting, so the paper's scoped empirical claims stand. But the condition that the reader already placed on the verdict remains necessary. The proposed boundary manipulations are cheap and would settle whether the boundary oracle is the active ingredient. I therefore recommend no change to the CONDITIONAL verdict; the condition should be stated in terms of the boundary-gating flag as well as the label ordering.

The referee's report

If the empirical claims hold, TeReL is a useful demonstration that a spatially and temporally local, batch-free self-supervised rule can learn a non-trivial representation in a small network, and it connects modern regularized self-supervised learning to SFA, trace learning, and Hebbian/anti-Hebbian synaptic rules. The paper has clear strengths: the gradient computation in Proposition 1 is clean; the official test split is held out; model selection is on a separate validation split; results are reported with paired confidence intervals; and code and a DOI are provided. The ablations also behave as expected for a VICReg-style objective, with collapse and divergence in the intended control conditions. However, the central evidence for the temporal signal is confounded by the boundary flag I_t, and the lateral-correction component of the algorithm is not derived from the stated objective. The significance for natural, unsegmented streams is therefore not yet established, although the authors do acknowledge the label-ordered stream as a limitation in Section 6.

Pith's reading of the paper (a machine's summary, not the authors')

The paper introduces Temporal Regularized Learning (TeReL), a self-supervised objective for sequential input whose update is local in both space and time: every neuron uses only its own pre- and postsynaptic activity plus a small amount of detached state, with no backward pass between layers and no gradient flowing through the past. The central claim is that a VICReg-style triple of invariance, variance, and covariance regularizers can be instantiated per neuron, using the previous observation as the positive pair and running statistics as fixed state, and that this is enough to learn a useful representation in a small network. Trained with batch size one on a stream of class-homogeneous MNIST chunks, a two-layer fully connected encoder reaches $96.48\%$ linear-probe accuracy against $95.09\%$ for a random encoder and about $90\%$ for classical SFA; a variant that relaxes both locality constraints reaches $98.41\%$, on par with supervised backpropagation at $98.44\%$. Because training and inference use the same per-sample computation, the rule can keep learning from unlabeled data after training without losing accuracy. The practical point is that temporally local, batch-free rules with bounded per-neuron memory fit streaming and energy-constrained hardware.

What carries the argument. The load-bearing object is the per-neuron regularized target. For observation $t$ the residual $$r_t = I_t(z_t - p_t) - \frac{\lambda_V}{\lambda_T} g_t \odot (z_t - \mu_t) + \frac{\lambda_C}{2\lambda_T} A_t (z_t - \mu_t), \qquad A_t = \mathrm{offdiag}(M_t),$$ combines temporal invariance (pull the activation $z_t$ toward the previous activation $p_t$, gated by sequence validity $I_t$), variance expansion ($g_t = [\tau - v_t]_+$), and a lagged covariance penalty. The residual defines a detached activation target $\hat{z}_t = \mathrm{sg}(z_t - r_t)$; through a ReLU it becomes the preactivation residual $b_t = r_t \odot \mathbb{1}[a_t > 0]$, and one matrix–vector pass through the lateral matrix gives the neuron state $s_t = b_t - \eta\kappa M_t b_t$. Feedforward weights update as $W_{t+1} = (1 - \alpha\lambda_W)W_t - \alpha c\, s_t x_t^\top$, so each synapse sees one presynaptic scalar and one postsynaptic scalar, the Hebbian form; the lateral matrix updates as $M_{t+1} = \rho M_t + (1-\rho)\left[(1-\omega)\,\mathrm{offdiag}(\bar{z}_t \bar{z}_t^\top) + \omega s_t s_t^\top\right]$, whose negative is anti-Hebbian: same-signed states make the lateral connection more inhibitory. Persistent state is bounded: three scalars per neuron (previous activation, running mean, running variance) plus one $D \times D$ lateral matrix per layer, and plain SGD adds no optimizer state. This machinery converts a batch-level self-supervised objective into a causal, per-neuron, online loss with no backward pass and no gradient through the history that produced the state.

Core claim. TeReL is a soft, causal form of Slow Feature Analysis expressed as a per-neuron loss. The temporal term penalizes the squared difference between consecutive activations, the discrete-time derivative that SFA minimizes; the variance term expands coordinates whose running variance lies below a target $\tau$; and the covariance term applies a lagged decorrelation signal through a lateral matrix $M_t$. Running statistics and the previous activation are non-differentiated state, so the gradient at a preactivation is a signed neuron state $s_t$ that drives a Hebbian feedforward update (postsynaptic state times presynaptic input) and an anti-Hebbian lateral update. At batch size one on a stream of randomly permuted class-homogeneous chunks of 16 MNIST samples, the two-layer encoder reaches $96.48 \pm 0.15\%$ held-out linear-probe accuracy, a paired gain of 1.39 points over a random encoder and about 6.4 points over batch and incremental SFA. Every ingredient is shown to matter: shuffling the stream costs 1.08 paired points, removing the temporal term costs 1.50 points, removing the variance term collapses the representation (median feature variance near $1.4 \times 10^{-5}$, accuracy $15.36\%$), and removing the covariance term makes training diverge. When the same objective is optimized with backpropagation through subsequences of 128 observations, accuracy reaches $98.41 \pm 0.04\%$, on par with the fully supervised network at $98.44 \pm 0.09\%$, so the gap to backpropagation is attributed to the locality constraints rather than to the objective. Continued unlabeled updates on the evaluation stream, with the step scaled by $10^{-3}$, hold accuracy at $96.50\%$ versus $96.48\%$ for the fixed encoder while the feedforward weights keep changing ($\|\Delta W\|_2 = 0.047$).

The premise it rests on. The load-bearing premise is that temporally adjacent observations share the slow feature the representation should capture: in the experiments this is injected by sorting MNIST into class-homogeneous chunks of 16, so adjacency means same label, and Section 6 acknowledges that natural streams contain several time scales, so the same gain on video or sensor streams is not established.

If the paper is right (Pith's inferences, not the authors')

Self-supervised learning can run on a stream with batch size one: large batches, stored negatives, and cross-sample statistics are replaced by temporally adjacent positives plus per-neuron running statistics, cutting the per-layer memory to $3D$ scalars and one $D \times D$ lateral matrix.

Training and inference use the same per-sample computation, so a deployed encoder can keep learning from unlabeled observations: at a $10^{-3}$-scaled step, continued updates hold accuracy at $96.50\%$ against $96.48\%$ for the fixed encoder while the feedforward weights keep changing.

Each loss term and the temporal order carry weight: removing the temporal term costs 1.50 paired points, shuffling the stream costs 1.08 points, removing the variance term collapses the representation, and removing the covariance term makes training diverge, so the rule is not reducible to a single mechanism.

The objective, not the locality, sets the ceiling in these experiments: TeReL-Offline, which backpropagates the same three-term loss through subsequences, reaches $98.41\%$, on par with the fully supervised network at $98.44\%$.

TeReL positions modern regularized self-supervision as a descendant of Slow Feature Analysis and trace learning, and the paper argues on ablation evidence that temporal local rules gain from explicit variance control and decorrelation.

My inference: if temporal coherence is what carries the gain, TeReL should learn whichever latent variable is genuinely slow on a natural stream, such as pose or identity in video, and a linear probe on that slow variable rather than on the class label is the right test; the paper itself names video and sensor streams as the key extension.

My inference: the chunked-MNIST setting aligns the slow variable exactly with the class label, so a fixed coefficient balance $(\lambda_T, \lambda_V, \lambda_C)$ may need to be scale-adaptive on natural streams, since the ablations show the outcome is sensitive to the ordering and to each term.

My inference: the $D^2$ lateral matrix dominates memory at width, so low-rank, sparse, or structured lateral matrices, which the paper flags as future work, are the natural test of whether decorrelation can be preserved at lower cost.

My inference: because training and inference share one computation with bounded state, TeReL is a candidate for neuromorphic and analog implementation; a testable prediction is that the signed neuron state $s_t$ alone is the information a synapse must read to update its weight.

Check this review

Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.

How often reviews like this one are right

Reviews like this one: written by deepseek-v4-flash, where what was read was not recorded.

How often objections from reviews like this one are right: not yet measured

Every value above, in the review's public record (JSON)

Reviews by people

This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.

Request a human review

How Pith checks papers · The full audit of this paper

The paper

Abstract

Layer-local self-supervised objectives remove the backward pass between layers, but almost all still compute their loss over a batch of samples and are therefore local in space but not in time. We introduce Temporal Regularized Learning (TeReL), a spatially and temporally local learning rule for self-supervised representation learning from sequential input streams. The key idea is to instantiate a VICReg-style objective with temporally adjacent positives and non-differentiated running population statistics, yielding a per-neuron online loss with bounded state. The objective is a soft form of Slow Feature Analysis, and its gradient is a Hebbian update of the feedforward weights and an anti-Hebbian update of one lateral matrix per layer. The aim of the experiments is to show that the rule works in a small network, not to compete with backpropagation. On a class-chunked MNIST stream, a two-layer fully connected network trained with batch size one outperforms random features (96.48% versus 95.09% linear-probe accuracy) and classical SFA, and a variant that relaxes both locality constraints is comparable to supervised backpropagation. Because training and inference use the same computation, the network can keep learning from unlabeled data during inference without loss of accuracy.

The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.

abstract compiled · 2026-11-13

The authors' main claim, as the classifier picked it out 0.90

“On a class-chunked MNIST stream, a two-layer fully connected network trained with batch size one outperforms random features (96.48% versus 95.09% linear-probe accuracy) and classical SFA, and a variant that relaxes both locality constraints is comparable to supervised backpropagation.”

How the main claim is stated: asserted 1.00

What the abstract itself states, as probabilities: claims first 0.12 · states limitation 0.95 · human participants 0.02 · controlled comparison 0.94 · randomized assignment 0.02 · states quantitative result 0.99 · compares with prior results 0.98 · mentions code or data availability 0.03

The abstract, sentence by sentence

This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.

SentenceRoleThe authors' own workStates the main conclusionStrength
Layer-local self-supervised objectives remove the backward pass between layers, but almost all still compute their loss over a batch of samples and are therefore local in space but not in time. background 0.59 0.09 0.04 asserted 1.00
We introduce Temporal Regularized Learning (TeReL), a spatially and temporally local learning rule for self-supervised representation learning from sequential input streams. contribution 1.00 0.88 0.27 asserted 1.00
The key idea is to instantiate a VICReg-style objective with temporally adjacent positives and non-differentiated running population statistics, yielding a per-neuron online loss with bounded state. method 0.94 0.82 0.11 asserted 1.00
The objective is a soft form of Slow Feature Analysis, and its gradient is a Hebbian update of the feedforward weights and an anti-Hebbian update of one lateral matrix per layer. method 0.98 0.81 0.07 asserted 1.00
The aim of the experiments is to show that the rule works in a small network, not to compete with backpropagation. limitation 0.94 0.39 0.05 asserted 0.98
On a class-chunked MNIST stream, a two-layer fully connected network trained with batch size one outperforms random features (96.48% versus 95.09% linear-probe accuracy) and classical SFA, and a variant that relaxes both locality constraints is comparable to supervised backpropagation. result 1.00 0.96 0.37 asserted 1.00
Because training and inference use the same computation, the network can keep learning from unlabeled data during inference without loss of accuracy. interpretation 0.58 0.81 0.20 asserted 0.98

What kind of paper the classifier reports this is: method or tool 1.00 · evidence from observation or experiment 0.57

What was examined, and what was not

Sources read

  • arXiv listing abstract v1

Not examined

  • full text
  • figures and tables
  • references and citing sentences
  • relations to other papers

Levels run

  • P abstract

Not run

  • S sections
  • T body sentences
  • C citations
  • R cross-paper relations

How the labels were made

Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-11-13: 1 request, 5,771 input tokens.

Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON

Figures (4)
Figure 1 of the paper
Figure 1. TEREL in one layer. a, The temporal, variance, and covariance terms form a regularized target. b, Its residual and one pass through the lateral matrix define the neuron state. c, The state drives a Hebbian feedforward and an anti-Hebbian lateral update. 1 Introduction Learning from unlabeled sensory streams is a practical necessity for both biological modeling and energy-efficient machine learning on constrained hardware. Modern self-supervised methods succeed arXiv:2611.17019v1 [cs.LG] 12 Nov 2026
Figure 2 of the paper
Figure 2. Representation quality and paired temporal effects on MNIST. a
Figure 3 of the paper
Figure 3. Class structure in the MNIST TEREL representation. a, Laplacian embedding of validation features of both layers; colors denote digits and are used only for visualization. b, Fraction of same-class neighbors in the complete 768-dimensional representation. The dotted line is the chance level. We probe whether TEREL yields class-structured latent codes suitable for a simple linear readout. The Laplacian embedding in Figure 3a shows class-consistent branches rather than a single collapsed mass. Because a two-dimensional projection can distort neighborhoods, Figure 3b measures the same…
Figure 4 of the paper
Figure 4. Signed postsynaptic state. a, State trajectories of four randomly selected second-layer neurons over the same interval of the validation stream. b, Their state distributions over the complete validation stream; black bars mark medians. 8

Figures are the authors', reproduced from the paper.

References (33)
  1. [1] Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A Simple Framework for Contrastive Learning of Visual Representations.arXiv preprint arXiv:2002.05709, 2020 on PithPith's reviewarXiv
  2. [2] Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Representation Learning with Contrastive Predictive Coding.arXiv preprint arXiv:1807.03748, 2018 on PithPith's reviewarXiv
  3. [3] Adrien Bardes, Jean Ponce, and Yann LeCun. VICReg: Variance-Invariance-Covariance Regular- ization for Self-Supervised Learning. InInternational Conference on Learning Representations, 2022. URL https://openreview.net/forum?id=xm6YD62D1Ub. 9 on Pith
  4. [4] Jure Zbontar, Li Jing, Ishan Misra, Yann LeCun, and Stéphane Deny. Barlow Twins: Self- Supervised Learning via Redundancy Reduction.arXiv preprint arXiv:2103.03230, 2021 on PithPith's reviewarXiv
  5. [5] David E Rumelhart, Geoffrey E Hinton, and Ronald J Williams. Learning representations by back-propagating errors.nature, 323(6088):533–536, 1986 on Pith
  6. [6] Rajesh P . N. Rao and Dana H. Ballard. Predictive Coding in the Visual Cortex: A Functional Interpretation of Some Extra-Classical Receptive-Field Effects.Nature Neuroscience, 2(1):79–87, on Pith
  7. [7] Benjamin Scellier and Yoshua Bengio. Equilibrium propagation: Bridging the gap between energy-based models and backpropagation.Frontiers in computational neuroscience, 11:24, 2017 on Pith
  8. [8] Giacomo Indiveri and Shih-Chii Liu. Memory and information processing in neuromorphic systems.Proceedings of the IEEE, 103(8):1379–1397, 2015 on Pith
  9. [9] Mike Davies, Narayan Srinivasa, Tsung-Han Lin, Gautham Chinya, Yongqiang Cao, Sri Harsha Choday, Georgios Dimou, Prasad Joshi, Nabil Maghazeh, Shirley Miao, and other. Loihi: A neuromorphic manycore processor with on-chip learning.IEEE Micro, 38(1):82–99, 2018 on Pith
  10. [10] Laurenz Wiskott and Terrence J Sejnowski. Slow feature analysis: Unsupervised learning of invariances.Neural computation, 14(4):715–770, 2002 on Pith
  11. [11] P Földiák. Learning invariance from transformation sequences.Neural computation, 3(2):194–200, 1991 on Pith
  12. [12] Cengiz Pehlevan, Tao Hu, and Dmitri B. Chklovskii. A Hebbian/Anti-Hebbian Neural Network for Linear Subspace Learning: A Derivation from Multidimensional Scaling of Streaming Data. Neural Computation, 27(7):1461–1495, 2015. doi: 10.1162/NECO_a_00745 on Pithdoi:10.1162/neco_a_00745
  13. [13] Cengiz Pehlevan, Anirvan M. Sengupta, and Dmitri B. Chklovskii. Why Do Similarity Matching Objectives Lead to Hebbian/Anti-Hebbian Networks?Neural Computation, 30(1):84–124, 2018. doi: 10.1162/neco_a_01018 on Pithdoi:10.1162/neco_a_01018
  14. [14] Erkki Oja. A simplified neuron model as a principal component analyzer.Journal of mathematical biology, 15(3):267–273, 1982 on Pith
  15. [15] Sindy Löwe, Peter O’Connor, and Bastiaan S. Veeling. Putting An End to End-to-End: Gradient- Isolated Learning of Representations. InAdvances in Neural Information Processing Systems, vol- ume 32, 2019 on Pith
  16. [16] Yuwen Xiong, Mengye Ren, and Raquel Urtasun. LoCo: Local Contrastive Representation Learning. InAdvances in Neural Information Processing Systems, volume 33, 2020 on Pith
  17. [17] Bernd Illing, Jean Ventura, Guillaume Bellec, and Wulfram Gerstner. Local plasticity rules can learn deep representations using self-supervised contrastive predictions. InAdvances in Neural Information Processing Systems, volume 34, 2021 on Pith
  18. [18] Shoaib Ahmed Siddiqui, David Krueger, Yann LeCun, and Stéphane Deny. Blockwise Self- Supervised Learning at Scale.Transactions on Machine Learning Research, 2024 on Pith
  19. [19] Joy Datta, Rawhatur Rabbi, Puja Saha, Aniqua Nusrat Zereen, M. Abdullah Al-Wadud, and Jia Uddin. Deep Representation Learning Using Layer-Wise VICReg Losses.Scientific Reports, 15: 27049, 2025. doi: 10.1038/s41598-025-08504-2. 10 on Pithdoi:10.1038/s41598-025-08504-2
  20. [20] Geoffrey E Hinton. The Forward-Forward Algorithm: Some Preliminary Investigations.arXiv preprint arXiv:2212.13345, 2022 on PithPith's reviewarXiv
show 13 more
  1. [21] Yann LeCun, Sumit Chopra, Raia Hadsell, Marc’Aurelio Ranzato, and Fu Jie Huang. A tutorial on energy-based learning.Predicting structured data, 1(0), 2006 on Pith
  2. [22] Antti Tarvainen and Harri Valpola. Mean Teachers Are Better Role Models: Weight-Averaged Consistency Targets Improve Semi-Supervised Deep Learning Results. InAdvances in Neural Information Processing Systems, volume 30, 2017 on Pith
  3. [23] Jean-Bastien Grill, Florian Strub, Florent Altché, Corentin Tallec, Pierre H. Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhaohan Daniel Guo, Mohammad Gheshlaghi Azar, Bilal Piot, koray kavukcuoglu, Remi Munos, and Michal Valko. Bootstrap Your Own Latent:... on PithPith's reviewarXiv
  4. [24] Xinlei Chen and Kaiming He. Exploring Simple Siamese Representation Learning.arXiv preprint arXiv:2011.10566, 2021 on PithPith's reviewarXiv
  5. [25] Zhizhong Li and Derek Hoiem. Learning without Forgetting. InEuropean Conference on Computer Vision, 2016 on Pith
  6. [26] James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A. Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, Demis Hassabis, Claudia Clopath, Dharshan Kumaran, and Raia Hadsell. Overcoming Catastrophic Forgetting i... on Pith
  7. [27] Davide Wiest. Improved Elastic Weight Consolidation as an Optimization Constraint for Continual Learning. https://axym.org/work/iewc/, 2026 on Pith
  8. [28] Varun Raj Kompella, Matthew Luciw, and Jürgen Schmidhuber. Incremental Slow Feature Analy- sis: Adaptive and Episodic Learning from High-Dimensional Input Streams.Neural Computation, 24(11):2994–3024, 2012. doi: 10.1162/NECO_a_00344 on Pithdoi:10.1162/neco_a_00344
  9. [29] David Lipshutz, Charles Windolf, Siavash Golkar, and Dmitri Chklovskii. A Biologically Plausible Neural Network for Slow Feature Analysis. InAdvances in Neural Information Processing Systems, volume 33, pages 14986–14996, 2020 on Pith
  10. [30] Simon M Stringer, G Perry, Edmund T Rolls, and JH Proske. Invariant object recognition in the visual system with novel views of 3D objects.Neural Networks, 15(10):1235–1246, 2002 on Pith
  11. [31] Peter Földiák. Forming sparse representations by local anti-Hebbian learning.Biological cybernetics, 64(2):165–170, 1990 on Pith
  12. [32] Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition.Proceedings of the IEEE, 86(11):2278–2324, 1998 on Pith
  13. [33] Prannay Khosla, Piotr Teterwak, Chen Wang, Aaron Sarna, Yonglong Tian, Phillip Isola, Aaron Maschinot, Ce Liu, and Dilip Krishnan. Supervised Contrastive Learning. InAdvances in Neural Information Processing Systems, volume 33, pages 18661–18673, 2020. 11 Table 1:Encoder confi... on Pith

Challenges and discussion

Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.

No comments yet.

Sign in to comment