Pith. sign in

REVIEW 3 major objections 5 minor 40 references

DeInfoReg: A Decoupled Learning Framework for Better Training Throughput

T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper introduces DeInfoReg, a decoupled training framework that splits a deep network into independently updated modules with local variance-invariance-covariance losses, claiming accuracy comparable to or better than backpropagation…

desk verdict A credible accuracy result for decoupled learning, but the throughput claim is not supported without a data-parallel BP baseline. read the letter →

arxiv 2506.18193 v2 pith:RFKTTDNB submitted 2025-06-22 cs.LG cs.AIcs.DC

Zih-Hao Huang , You-Teng Lin , Hung-Hsuan Chen This is my paper · ORCID
classification cs.LGcs.AIcs.DC
keywords decoupledlearninglocallossgradienttruncationmodelparallelismvanishingcontrastivevariance-invariance-covarianceregularizationtrainingthroughput
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

DeInfoReg is a way to train deep neural networks without one long end-to-end gradient path. The network is split into modules, each with its own classifier and a local loss made of variance, invariance, and covariance terms; gradients are blocked at module boundaries. The authors claim this keeps gradient signals strong in deep networks, gives accuracy comparable to or better than standard backpropagation on most tested NLP and vision datasets, and makes the modules independent enough to run as a pipeline across GPUs. If true, it would mean deep models can be trained stably at larger depths, with smaller batch sizes, and with meaningful multi-GPU speedups instead of strictly sequential forward-backward passes.

What carries the argument

The load-bearing mechanism is gradient truncation combined with a three-term local loss: a variance term that penalizes embedding dimensions with low spread (preventing collapse), an invariance term that matches the pairwise similarity structure of embeddings to the pairwise similarity of one-hot labels, and a covariance term that de-correlates embedding dimensions. Each module also has a small classifier trained with cross-entropy, weighted by $\alpha = 0.001$ relative to the local loss. Because each module's gradients stop at its own boundary, the modules can compute forward and backward concurrently in a pipeline, which is what converts the architecture into a model-parallel training scheme.

What would settle it

Run data-parallel backpropagation with the same VGG/CIFAR-100 setup on two and four GPUs and compare training time per epoch to pipelined DeInfoReg on the same hardware; if data-parallel BP matches or beats those times, the throughput claim fails. A second check is to train DeInfoReg with each of the three local-loss terms removed in turn on a third dataset; the paper's ablation shows all three improve accuracy, so a dataset where removing one term does not hurt would weaken that part of the design story.

Watch

Extended reading notes

Core claim

The central claim is that decomposing a deep network into locally supervised modules, each trained by its own combination of cross-entropy and a variance-invariance-covariance regularization loss with gradient flow truncated between modules, removes the need for a single long backpropagation path. On IMDB, AGNews, DBpedia, CIFAR-100, and TinyImageNet across LSTM, Transformer, VGG, and ResNet, DeInfoReg matches or exceeds BP accuracy over nearly all batch sizes; BP remains better only on CIFAR-10. Table 5 shows BP accuracy collapses as depth passes eight LSTM blocks while DeInfoReg stays stable to fourteen, and Figure 3 shows near-zero input-side gradients for BP but not for DeInfoReg. Table 7 reports that pipelined DeInfoReg trains 30 to 47 percent faster than single-GPU BP at the same batch size, at the cost of being 10 to 19 percent slower than BP on one GPU. The paper also claims better tolerance to noisy labels and roughly equal GPU memory usage.

Load-bearing premise

The throughput advantage rests on comparing DeInfoReg running on two or four GPUs against standard backpropagation running on a single GPU; if backpropagation were run in data-parallel mode on the same two or four GPUs and scaled near-linearly, the reported speedups would not show an advantage, and DeInfoReg on one GPU is actually slower than BP.

Editorial extensions

If this is right

  • Deep networks can be trained stably at depths where standard backpropagation fails: DeInfoReg holds roughly 92 percent accuracy on AGNews LSTM at depths 4 through 14, while BP drops to 25 percent at depth 14.
  • Batch size no longer drives accuracy: DeInfoReg's accuracy varies little from batch size 64 to 512, so small-batch training can save GPU memory.
  • Noisy-label robustness improves: at 40 percent label noise DeInfoReg keeps 81.28 percent accuracy versus BP's 76.08 percent on IMDB.
  • Pipeline execution of decoupled modules shortens per-iteration time: DeInfoReg reports 1.30 to 1.35 times speedup on 2 GPUs and 1.41 to 1.47 times on 4 GPUs relative to single-GPU BP.
  • The modular design is a natural fit for pipeline parallelism, reducing per-iteration time from 17 to 8 time units in the illustrative four-module schedule.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper does not compare against data-parallel backpropagation using the same number of GPUs; an editor-level test would measure whether the 1.47 times speedup is a property of decoupling or simply of using four GPUs.
  • Because DeInfoReg places a classifier on every module, the same architecture could support early-exit inference at test time, a use the paper does not explore.
  • The invariance term, which aligns embedding similarities to one-hot label similarities, is doing most of the work in the ablation; a natural extension is to test whether softer label distributions improve the transfer of inter-class structure.
  • If the variance and covariance terms indeed prevent collapse without large batches, the same loss stack could be dropped into other decoupled or locally supervised models; the paper only demonstrates it inside DeInfoReg.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. This paper introduces DeInfoReg, a decoupled learning framework that splits a deep network into modules, truncates gradient flow between modules, and trains each module with a local loss combining variance, invariance, and covariance terms (inspired by VICReg) plus a cross-entropy term. The authors evaluate DeInfoReg against backpropagation (BP), Associated Learning (AL), and Supervised Contrastive Parallel Learning (SCPL) on NLP (LSTM, Transformer) and CV (VGG, ResNet) benchmarks, reporting that DeInfoReg achieves the best accuracy on most datasets, particularly on harder datasets, while being robust to label noise and to network depth. The paper further claims that the decoupled architecture enables pipeline model parallelism across multiple GPUs, with reported speedups of 1.30x–1.47x over single-GPU BP.

Significance. The paper addresses an important problem: mitigating vanishing gradients and enabling model parallelism in deep neural network training. The empirical evaluation is broad (three NLP and three CV datasets, four architectures), and the results on CIFAR-100 and TinyImageNet are strong, with DeInfoReg consistently outperforming BP and other decoupled baselines on those tasks. The code is released, which supports reproducibility. The noisy-label robustness and depth-stability results are also valuable. However, the central throughput claim—the one in the paper's title—is not supported by the experiments as reported, because the multi-GPU speedups are measured only against single-GPU BP without a data-parallel BP baseline. This is a significant gap that must be addressed before the claims can be accepted.

major comments (3)
  1. [Section 4.5, Table 7] The throughput claim is not established because the baselines use only a single GPU. The paper reports speedups of DeInfoReg on 1, 2, and 4 GPUs relative to BP running on a single GPU (e.g., 0.81x, 1.30x, 1.41x for batch size 256). The proper comparison is against data-parallel BP using the same number of GPUs and the same total batch size. If data-parallel BP scales at a typical rate, its 4-GPU speedup over single-GPU BP would likely exceed DeInfoReg's 1.47x, and the headline 'better training throughput' would not hold. The paper must add this baseline or significantly reframe the claim as 'enables pipeline parallelism for models that are hard to parallelize with data parallelism' and provide evidence in that setting.
  2. [Abstract and Section 1] The claim that DeInfoReg 'achieves superior performance' over BP is contradicted by the paper's own results on CIFAR-10 (Tables 3 and 4), where BP outperforms DeInfoReg on both VGG and ResNet (e.g., ResNet/CIFAR-10: 93.55 ± 0.12 vs 91.96 ± 0.19 at batch 512). The abstract should be qualified to match the nuanced statement in Section 5, which correctly notes that DeInfoReg may perform worse on some datasets.
  3. [Section 3.3, Equations (4)–(8) and Algorithm 1] The mathematical definitions of the local loss are inconsistent with the PyTorch pseudocode. The variance loss in Eq. (4) is defined as (1/C) max(0, γ - S(Ê_l)), while Algorithm 1 computes var_loss = torch.mean(F.relu(1 - std_x)) / batch_size; the threshold γ is never specified in the text, and the division by batch size is not present in the equation. The invariance loss in Eq. (5) is (1/N)||Sim(Y) - Sim(Norm(Ê_l))||^2, whereas the code uses F.mse_loss, which averages over all N^2 entries. These discrepancies make the exact objective ambiguous and impede reproducibility; they must be resolved.
minor comments (5)
  1. [Equation (2)] There is a typo: the last component of the mean vector is written as (1/N) Σ e_{i,N}^l but should be e_{i,C}^l, since C is the number of columns/features.
  2. [Section 4.5] The description of the speedup experiment is sparse: no information is given about how modules are assigned to GPUs, whether micro-batching is used, how parameter synchronization is handled, or how communication overhead is included. This makes it difficult to interpret the measured speedups.
  3. [Section 5] The paper's Discussion mentions that 'many default settings ... are primarily designed for standard BP and may not be optimal for decoupled models.' This is a useful caveat, but the abstract and introduction do not reflect this caveat; aligning the abstract with this statement would improve the paper.
  4. [Table 5] The BP accuracy at L=14 is reported as '25 ± 0.24', which is close to random for AGNews (4 classes); this dramatic drop deserves a brief comment in the text, for example confirming training instability rather than a metric artifact.
  5. [Section 2.3] The comparison with pipeline-parallelism methods (GPipe, PipeDream) is brief; adding a few sentences on how DeInfoReg's pipelining differs from these in terms of gradient locks and communication patterns would help situate the contribution.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: DeInfoReg's accuracy, robustness, and throughput claims are supported by in-paper experiments against independent baselines, and its loss terms are adopted from external work (VICReg) rather than fitted to the evaluated data.

full rationale

The paper's central claims are empirical and self-contained, so no circular step is present. The per-module local loss in Equations 4-8 combines variance, invariance, and covariance terms adopted from the external VICReg framework (Bardes et al., 2021) and applied locally; these terms are not defined in terms of the test accuracies they are used to claim, so there is no self-definitional reduction. The accuracy results in Tables 1-4 are genuine out-of-sample predictions on open datasets (IMDB, AGNews, DBpedia, CIFAR-10, CIFAR-100, TinyImageNet) compared against independent baselines (BP, AL, SCPL), and no fitted parameter is later relabeled as a prediction; hyperparameters such as alpha and gamma are tuned on validation data, which is standard practice and not circular. The decoupled-paradigm literature is cited from the authors' own prior work (Kao and Chen 2021, Wu et al. 2021) and same-institution reports (Wang 2021, Ho 2022), but these citations are not load-bearing: the paper re-implements and re-benchmarks AL and SCPL in its own experiments, independently verifying the background premise in-paper, and no uniqueness theorem is imported to forbid alternatives. The throughput claim in Section 4.5 (Table 7) is the weakest point, but it is a baseline-selection and experimental-design issue, not circularity: DeInfoReg's 1.30x-1.47x speedup is measured against single-GPU BP, and the paper itself transparently reports that DeInfoReg on one GPU is 0.81x-0.90x of BP; the absence of a data-parallel BP baseline at 2-4 GPUs means the 'significantly improves training throughput' claim is not fully established, yet this does not reduce the method's output to its input by definition. The gradient-magnitude study (Section 4.3) and noise-robustness study (Table 6) are empirical observations rather than consequences encoded in the loss function. Overall, the derivation chain is non-circular.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

No new physical or conceptual entities are introduced; the loss components come from VICReg and the decoupled paradigm from AL/SCPL. The free parameters are standard training hyperparameters, with alpha tuned on validation and gamma not reported.

free parameters (2)
  • alpha (local loss weight) = 0.001 in main experiments; 1e-2 to 1e-3 optimal in Section 4.8
    Controls the trade-off between cross-entropy loss and local loss in Equation (9). Tuned on validation data and varied in the alpha study.
  • gamma (variance threshold) = not reported
    Threshold in the variance-loss hinge function (Equation 4). The paper does not state the value used, but the local loss depends on it.
assumptions (4)
  • domain assumption Local losses (variance, invariance, covariance) provide useful learning signals for each module.
    Borrowed from VICReg and assumed to prevent collapse and guide representation learning without end-to-end gradients, as stated in Section 3.1.
  • domain assumption Truncating gradient flow at module boundaries does not prevent the network from learning the global classification task.
    The whole method depends on this; Section 3.2 describes gradient truncation and the per-module losses without a theoretical argument that the global objective is preserved.
  • domain assumption The invariance loss computed on one-hot label similarities is a valid training target.
    Equation (5) aligns embedding similarities with label similarities; this assumes label space structure transfers to representation space.
  • domain assumption Pipeline scheduling overhead is small enough that the throughput gain is positive.
    Section 4.5 reports speedups, but the implementation details and synchronization costs are not described, and the missing multi-GPU BP baseline makes this assumption untested.

how reviews work

0 comments
Cite this review

Pith. "Pith review of DeInfoReg: A Decoupled Learning Framework for Better Training Throughput." pith.science (2026). https://pith.science/paper/RFKTTDNB

@misc{pith2026250618193,
  author       = {Pith},
  title        = {Pith review of: DeInfoReg: A Decoupled Learning Framework for Better Training Throughput},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RFKTTDNB}},
  note         = {Machine review of arXiv:2506.18193}
}
read the original abstract

This paper introduces Decoupled Supervised Learning with Information Regularization (DeInfoReg), a novel approach that transforms a long gradient flow into multiple shorter ones, thereby mitigating the vanishing gradient problem. Integrating a pipeline strategy, DeInfoReg enables model parallelization across multiple GPUs, significantly improving training throughput. We compare our proposed method with standard backpropagation and other gradient flow decomposition techniques. Extensive experiments on diverse tasks and datasets demonstrate that DeInfoReg achieves superior performance and better noise resistance than traditional BP models and efficiently utilizes parallel computing resources. The code for reproducibility is available at: https://github.com/ianzih/Decoupled-Supervised-Learning-for-Information-Regularization/.

Figures

Figures reproduced from arXiv: 2506.18193 by the authors.

Figure 1
Figure 1. A standard neural network (left) and the DeInfoReg network architecture (right). The green and red dashed [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. An illustrative example comparing the GPU usage per iteration for standard BP, naïve model parallelism [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. Gradient changes in the encoder output of 10 LSTM blocks (Top: DeInfoReg, Bottom: BP) [PITH_FULL_IMAGE:figures/full_fig_p012_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

40 extracted references · 25 canonical work pages

  1. [1]

    Vicreg: Variance-invariance-covariance regularization for self-supervised learning

    Adrien Bardes, Jean Ponce, and Yann LeCun. Vicreg: Variance-invariance-covariance regularization for self-supervised learning. arXiv preprint arXiv:2105.04906, 2021

    arXiv 2021

  2. [2]

    A simple framework for contrastive learning of visual representations

    Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In International conference on machine learning, pages 1597--1607. PMLR, 2020 a

    work page 2020

  3. [3]

    Exploring simple siamese representation learning

    Xinlei Chen and Kaiming He. Exploring simple siamese representation learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 15750--15758, 2021

    2021

  4. [4]

    Improved baselines with momentum contrastive learning

    Xinlei Chen, Haoqi Fan, Ross Girshick, and Kaiming He. Improved baselines with momentum contrastive learning. arXiv preprint arXiv:2003.04297, 2020 c

    arXiv 2003

  5. [5]

    Cover and Joy A

    Thomas M. Cover and Joy A. Thomas. Elements of Information Theory (Wiley Series in Telecommunications and Signal Processing). Wiley-Interscience, USA, 2006. ISBN 0471241954

    2006

  6. [6]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770--778, 2016

    2016

  7. [7]

    Momentum contrast for unsupervised visual representation learning

    Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual representation learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9729--9738, 2020

    2020

  8. [8]

    Realizing synchronized parameter updating, dynamic layer accumulation, and forward shortcuts in supervised contrastive parallel learning

    Ming-Yao Ho. Realizing synchronized parameter updating, dynamic layer accumulation, and forward shortcuts in supervised contrastive parallel learning. Technical report, National Central University, 2022

    work page 2022

Show all 40 references
  1. [9]

    The vanishing gradient problem during learning recurrent neural nets and problem solutions

    Sepp Hochreiter. The vanishing gradient problem during learning recurrent neural nets and problem solutions. International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems, 06 0 (02): 0 107--116, 1998

    1998

  2. [10]

    Gpipe: Efficient training of giant neural networks using pipeline parallelism

    Yanping Huang, Youlong Cheng, Ankur Bapna, Orhan Firat, Dehao Chen, Mia Chen, HyoukJoong Lee, Jiquan Ngiam, Quoc V Le, Yonghui Wu, et al. Gpipe: Efficient training of giant neural networks using pipeline parallelism. Advances in neural information processing systems, 32, 2019

    2019

  3. [11]

    Batch normalization: Accelerating deep network training by reducing internal covariate shift

    Sergey Ioffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In International conference on machine learning, pages 448--456. pmlr, 2015

    2015

  4. [12]

    Decoupled neural interfaces using synthetic gradients

    Max Jaderberg, Wojciech Marian Czarnecki, Simon Osindero, Oriol Vinyals, Alex Graves, David Silver, and Koray Kavukcuoglu. Decoupled neural interfaces using synthetic gradients. In International conference on machine learning, pages 1627--1635. PMLR, 2017

    2017

  5. [13]

    Scaling up visual and vision-language representation learning with noisy text supervision

    Chao Jia, Yinfei Yang, Ye Xia, Yi-Ting Chen, Zarana Parekh, Hieu Pham, Quoc Le, Yun-Hsuan Sung, Zhen Li, and Tom Duerig. Scaling up visual and vision-language representation learning with noisy text supervision. In International conference on machine learning, pages 4904--4916...

    2021

  6. [14]

    Beyond data and model parallelism for deep neural networks

    Zhihao Jia, Matei Zaharia, and Alex Aiken. Beyond data and model parallelism for deep neural networks. Proceedings of Machine Learning and Systems, 1: 0 1--13, 2019

    2019

  7. [15]

    Understanding dimensional collapse in contrastive self-supervised learning

    Li Jing, Pascal Vincent, Yann LeCun, and Yuandong Tian. Understanding dimensional collapse in contrastive self-supervised learning. arXiv preprint arXiv:2110.09348, 2021

    2021 arXiv

  8. [16]

    Associated learning: Decomposing end-to-end backpropagation based on autoencoders and target propagation

    Yu-Wei Kao and Hung-Hsuan Chen. Associated learning: Decomposing end-to-end backpropagation based on autoencoders and target propagation. Neural Computation, 33 0 (1): 0 174--193, 2021

    2021

  9. [17]

    Supervised contrastive learning

    Prannay Khosla, Piotr Teterwak, Chen Wang, Aaron Sarna, Yonglong Tian, Phillip Isola, Aaron Maschinot, Ce Liu, and Dilip Krishnan. Supervised contrastive learning. Advances in neural information processing systems, 33: 0 18661--18673, 2020

    2020

  10. [18]

    Deeply-supervised nets

    Chen-Yu Lee, Saining Xie, Patrick Gallagher, Zhengyou Zhang, and Zhuowen Tu. Deeply-supervised nets. In Artificial intelligence and statistics, pages 562--570. Pmlr, 2015

    2015

  11. [19]

    Self-organization in a perceptual network

    Ralph Linsker. Self-organization in a perceptual network. Computer, 21 0 (3): 0 105--117, 1988

    1988

  12. [20]

    Pipedream: Generalized pipeline parallelism for dnn training

    Deepak Narayanan, Aaron Harlap, Amar Phanishayee, Vivek Seshadri, Nikhil R Devanur, Gregory R Ganger, Phillip B Gibbons, and Matei Zaharia. Pipedream: Generalized pipeline parallelism for dnn training. In Proceedings of the 27th ACM symposium on operating systems principles, p...

    2019

  13. [21]

    Training neural networks with local error signals

    Arild N kland and Lars Hiller Eidnes. Training neural networks with local error signals. In International conference on machine learning, pages 4839--4850. PMLR, 2019

    2019

  14. [22]

    Representation learning with contrastive predictive coding

    Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748, 2018

    2018 arXiv

  15. [23]

    Self-supervised learning with an information maximization criterion

    Serdar Ozsoy, Shadi Hamdan, Sercan Arik, Deniz Yuret, and Alper Erdogan. Self-supervised learning with an information maximization criterion. Advances in Neural Information Processing Systems, 35: 0 35240--35253, 2022

    2022

  16. [24]

    Glove: Global vectors for word representation

    Jeffrey Pennington, Richard Socher, and Christopher D Manning. Glove: Global vectors for word representation. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pages 1532--1543, 2014

    2014

  17. [25]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pa...

    2021

  18. [26]

    Measuring the effects of data parallelism on neural network training

    Christopher J Shallue, Jaehoon Lee, Joseph Antognini, Jascha Sohl-Dickstein, Roy Frostig, and George E Dahl. Measuring the effects of data parallelism on neural network training. Journal of Machine Learning Research, 20 0 (112): 0 1--49, 2019

    2019

  19. [27]

    Spatiotemporal co-attention recurrent neural networks for human-skeleton motion prediction

    Xiangbo Shu, Liyan Zhang, Guo-Jun Qi, Wei Liu, and Jinhui Tang. Spatiotemporal co-attention recurrent neural networks for human-skeleton motion prediction. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44 0 (6): 0 3300--3315, 2021

    2021

  20. [28]

    Multi-granularity anchor-contrastive representation learning for semi-supervised skeleton-based action recognition

    Xiangbo Shu, Binqian Xu, Liyan Zhang, and Jinhui Tang. Multi-granularity anchor-contrastive representation learning for semi-supervised skeleton-based action recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45 0 (6): 0 7559--7576, 2022

    2022

  21. [29]

    Blockwise self-supervised learning at scale

    Shoaib Ahmed Siddiqui, David Krueger, Yann LeCun, and St \'e phane Deny. Blockwise self-supervised learning at scale. arXiv preprint arXiv:2302.01647, 2023

    2023 arXiv

  22. [30]

    Very deep convolutional networks for large-scale image recognition

    Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556, 2014

    2014 arXiv

  23. [31]

    Going deeper with convolutions

    Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. Going deeper with convolutions. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1--9, 2015

    2015

  24. [32]

    Coherence constrained graph lstm for group activity recognition

    Jinhui Tang, Xiangbo Shu, Rui Yan, and Liyan Zhang. Coherence constrained graph lstm for group activity recognition. IEEE transactions on pattern analysis and machine intelligence, 44 0 (2): 0 636--647, 2019

    2019

  25. [33]

    Branchynet: Fast inference via early exiting from deep neural networks

    Surat Teerapittayanon, Bradley McDanel, and Hsiang-Tsung Kung. Branchynet: Fast inference via early exiting from deep neural networks. In 2016 23rd international conference on pattern recognition (ICPR), pages 2464--2469. IEEE, 2016

    2016

  26. [34]

    On mutual information maximization for representation learning

    Michael Tschannen, Josip Djolonga, Paul K Rubenstein, Sylvain Gelly, and Mario Lucic. On mutual information maximization for representation learning. arXiv preprint arXiv:1907.13625, 2019

    1907 arXiv

  27. [35]

    Decomposing end-to-end backpropagation based on SCPL

    Cheng-Kai Wang. Decomposing end-to-end backpropagation based on SCPL . Technical report, National Central University, 2021

    2021

  28. [36]

    Revisiting locally supervised learning: an alternative to end-to-end training

    Yulin Wang, Zanlin Ni, Shiji Song, Le Yang, and Gao Huang. Revisiting locally supervised learning: an alternative to end-to-end training. The Ninth International Conference on Learning Representations, 2021

    2021

  29. [37]

    Associated learning: an alternative to end-to-end backpropagation that works on cnn, rnn, and transformer

    Dennis YH Wu, Dinan Lin, Vincent Chen, and Hung-Hsuan Chen. Associated learning: an alternative to end-to-end backpropagation that works on cnn, rnn, and transformer. In International Conference on Learning Representations, 2021

    2021

  30. [38]

    Higcin: Hierarchical graph-based cross inference network for group activity recognition

    Rui Yan, Lingxi Xie, Jinhui Tang, Xiangbo Shu, and Qi Tian. Higcin: Hierarchical graph-based cross inference network for group activity recognition. IEEE transactions on pattern analysis and machine intelligence, 45 0 (6): 0 6955--6968, 2020

    2020

  31. [39]

    Towards interpretable deep local learning with successive gradient reconciliation

    Yibo Yang, Xiaojie Li, Motasem Alfarra, Hasan Hammoud, Adel Bibi, Philip Torr, and Bernard Ghanem. Towards interpretable deep local learning with successive gradient reconciliation. Forty-first International Conference on Machine Learning, 2024

    2024

  32. [40]

    Barlow twins: Self-supervised learning via redundancy reduction

    Jure Zbontar, Li Jing, Ishan Misra, Yann LeCun, and St \'e phane Deny. Barlow twins: Self-supervised learning via redundancy reduction. In International conference on machine learning, pages 12310--12320. PMLR, 2021

    2021

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.