Pith. sign in

REVIEW 1 major objections 52 references

Improving Neural Network Training using Dynamic Learning Rate Schedule for PINNs and Image Classification

T0 review · 1 major / 0 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read DLRS adapts the learning rate each epoch from the normalized change in mini-batch losses, claiming faster and more stable training for PINNs and image classifiers.

desk verdict The DLRS idea is plausible, but the paper's own update rule cannot produce the reported effects—the experiments almost certainly used a different rule than the one specified. read the letter →

arxiv 2507.21749 v1 pith:FMEHCPRU submitted 2025-07-29 cs.CE cs.LG

D. Veerababu , Ashwin A. Raikar , Prasanta K. Ghosh This is my paper
classification cs.CEcs.LG
keywords dynamiclearningrateschedulingadaptationphysics-informedneuralnetworksHelmholtzequationimageclassificationMNISTCIFAR-10CNNtraining
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper proposes a learning-rate scheduler, DLRS, that changes the optimizer's learning rate at the end of each epoch based on how the mini-batch losses moved within that epoch. The authors aim to show that this loss-based rule removes the need to hand-tune or pre-schedule the learning rate, while making training faster and more stable than a fixed-rate Adam baseline. They test it on a physics-informed neural network for the 1-D Helmholtz equation and on image classification with MNIST and CIFAR-10, and report that DLRS keeps PINN relative errors below 1 percent and raises test accuracy on several CNN architectures. The intended contribution is a cheap, general layer that can sit on top of an existing optimizer and adjust a single scalar.

What carries the argument

The machinery is the normalized epoch-level loss slope $\Delta L_j = (L_j^{(B)} - L_j^{(1)}) / \bar{L}_j$, where $\bar{L}_j$ is the mean of the batch losses, combined with the order-of-magnitude scaling $n = \lfloor \log_{10}(\alpha_j) \rfloor$. The update rule $\alpha_{j+1} = \alpha_j - 10^n \, \delta_{\text{case}} \, \Delta L_j$ selects $\delta_{\text{case}}$ from three regimes, called divergent, flat, and convergent, and automatically produces a decrease when $\Delta L_j$ is positive and an increase when it is negative. This object carries the argument: it encodes whether the epoch ended worse than it started, normalized across datasets and architectures, and it determines the size of the rate change without adding gradient computations.

What would settle it

A decisive check is to run DLRS on the same architectures with shuffled batch order fixed across seeds and compare the learning-rate trajectory; if the sign of $\Delta L_j$ flips when batches are permuted, the scheduler is responding to ordering noise rather than to learning progress.

Watch

Extended reading notes

Core claim

DLRS treats the normalized difference between an epoch's first and last batch losses, $\Delta L_j = (L_j^{(B)} - L_j^{(1)}) / \bar{L}_j$, as a control signal. If $\Delta L_j > 1$, the learning rate is decreased through a factor involving $\delta_d = 0.5$; if $0 \leq \Delta L_j \leq 1$, the rate is mildly adjusted with $\delta_o = 1$; and if $\Delta L_j < 0$, the rate is increased with $\delta_i = 0.1$. The same update rule subtracts a scaled adjustment, so it lowers or raises the learning rate depending on the sign of $\Delta L_j$. The paper's central claim is that this single scalar signal is enough to recover stable convergence in PINN Helmholtz solves that otherwise diverge above 500 Hz and to accelerate loss decrease on MNIST and CIFAR-10, with reported CIFAR-10 test-accuracy gains such as +2.70 percent on VGG-19 and +3.12 percent on GoogLeNet. The authors present DLRS as a drop-in schedule over Adam and other base optimizers, requiring no extra gradient computations or line searches.

Load-bearing premise

The rule assumes that the normalized change from an epoch's first batch loss to its last batch loss reliably indicates whether the optimizer should slow down, stay put, or speed up, and that the chosen thresholds and factors (0.5, 1, 0.1) carry over to new problems without retuning.

Editorial extensions

If this is right

  • If DLRS works as reported, a user of Adam or another base optimizer can replace manual learning-rate decay with per-epoch loss-slope feedback at essentially zero extra cost.
  • PINN training for wave problems such as the 1-D Helmholtz equation could remain stable at frequencies where a fixed-rate loss diverges, with relative errors reported below 1 percent.
  • On classification benchmarks, deep networks that are sensitive to learning-rate tuning would gain accuracy without architecture changes; the paper reports gains on all five architectures it tests.
  • Because DLRS only touches the scalar learning rate, it should compose with existing acceleration techniques such as mixed precision and gradient accumulation without changing their behavior.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A consequence the authors leave implicit is that the same normalized-slope rule could serve as an online diagnostic: the fraction of epochs with positive $\Delta L_j$, or the size of the required decreases, could reveal whether a training run is fundamentally unstable before it finishes.
  • Because the control signal is a ratio, the rule might transfer to tasks with very different loss scales, such as language modeling or reinforcement learning, but the fixed thresholds and factors would likely need recalibration, which the paper does not test.
  • The comparison to fixed-rate training could be sharpened by testing DLRS against established schedules like cosine annealing and exponential decay, since the paper acknowledges those strategies but only benchmarks against Adacomp.
  • Since the update depends on the first and last batch losses, batching order and shuffling may materially affect the signal; a natural extension is measuring the sensitivity of DLRS to batch permutation, which the paper does not report.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

1 major / 0 minor

Summary. The manuscript proposes DLRS (Dynamic Learning Rate Scheduler), a heuristic that adjusts the learning rate once per epoch based on the normalized slope of mini-batch losses within that epoch. Three regimes are defined: a sharp loss increase triggers a decrease, a plateau triggers a small adjustment, and a steady decrease triggers an increase in the learning rate. The method is evaluated on a 1-D Helmholtz problem solved with a PINN, on MNIST and CIFAR-10 image classification with CNNs, and on five modern CNN architectures on CIFAR-10. The authors report that DLRS accelerates training and improves stability compared with fixed-rate Adam and with the Adacomp scheduler, including relative PINN errors below 1% and test-accuracy gains of up to +3.12% on GoogLeNet.

Significance. If the DLRS update rule were correctly specified and reproducible, the contribution would be a simple, low-overhead learning-rate heuristic: it uses only scalar loss summaries, adds O(1) work per epoch, and is optimizer-agnostic in principle. The paper also deserves credit for spanning PINNs and image classification and for including a comparison against Adacomp. However, the central algorithm as written in Section 3.2 and Algorithm 1 is internally inconsistent: the use of 10^{floor(log10 alpha_j)} rather than alpha_j makes the adjustment not proportional to the current learning rate, cannot produce the described halving/increasing behavior, and can even drive the learning rate negative without any clamping rule. In addition, the PINN hyperparameters in Section 5.1 are 'determined from the experiments', the classification results in Section 5.4 and Table 1 rest on a single random seed with no error bars, and no code is provided. These issues prevent the empirical claims from being verified as stated.

major comments (1)
  1. [3.2] The update rule in Section 3.2 and Algorithm 1 is internally inconsistent. The paper defines alpha_delta_j = 10^{floor(log10 alpha_j)} * delta_case * Delta L_j and then sets alpha_{j+1} = alpha_j - alpha_delta_j. This is not 'scaled proportionally to the current learning rate' as claimed. For any alpha_j in [0.001, 0.01), 10^{floor(log10 alpha_j)} = 0.001, so alpha_j = 0.005 and alpha_j = 0.009 receive the same absolute adjustment even though their relative adjustments differ by a factor of 1.8. With the stated delta_d = 0.5, a loss increase of Delta L_j = 1 reduces alpha_j = 0.005 to 0.0045, a 10% reduction rather than the intended halving; with delta_i = 0.1, an improving epoch changes alpha_j only negligibly. When alpha_j is a power of 10, e.g., alpha_j = 0.01 with Delta L_j = 2.5, alpha_delta_j = 0.0125 exceeds alpha_j, driving the learning rate negative, and no clamping or positivity constraint is specified. Since every reported result depends on this rule, the formula must be corrected (for example to alpha_delta_j = alpha_j * delta_case * Delta L_j), a clamping rule must be added, or the actual implementation must be provided; as written, Algorithm 1 cannot reproduce the described experiments.

Circularity Check

1 steps flagged · score 2.0 of 10

Mild tuning on the evaluation task; no construction-level circularity in the main derivation.

  1. fitted input called prediction [Section 5.1 (PINNs), hyperparameter statement and relative-error result]
    "The network is trained with hyperparameters that were determined from the experiments to be δd = 0.5 , δo = 1 (default) and δi = 0.1 in the DLRS algorithm with Adam optimizer. ... The relative error is observed to be less than 1% for all frequencies considered in the study."

    The three DLRS control factors are selected from the very PINN experiments that are then reported as evidence that DLRS stabilizes training and gives less than 1% relative error. To that extent the PINN demonstration is a fit to the test problem rather than an independent prediction. The circularity is limited: the update rule itself is not defined in terms of the final relative error, and the MNIST and CIFAR-10 comparisons provide separate empirical benchmarks using the same factors.

full rationale

The central DLRS claim is empirical rather than derived: Section 3.2 defines a normalized loss slope and a hand-specified piecewise update, and the paper then compares training with and without DLRS on PINNs, MNIST, and CIFAR-10. No load-bearing result is imported from the authors' prior work; the PINN formulation uses Lagaris trial solutions and Raissi losses, and the self-citation Veerababu and Ghosh (2024) merely names PINNs as a problem class. The only circularity-adjacent step is that δd, δo, and δi are stated to be 'determined from the experiments' in the same PINN section, so the reported <1% error is partly a tuned demonstration rather than a prediction. This is a mild fitted-input concern, not a derivation that reduces by construction. The reviewer's algorithmic inconsistency, namely that 10^n scaling is not actually proportional to α_j and can produce negative learning rates, is a correctness and implementation risk rather than a circularity of the derivation chain. Overall score 2.

Assumptions & free parameters 5 free parameters · 2 assumptions · 0 invented entities

The central claim rests on a small set of tuned hyperparameters (δd, δo, δi, initial rates) and two unproven assumptions about the relation between loss-slope and useful learning-rate changes. No new physical or mathematical entities are introduced. The main weakness is that the scheduler's effectiveness is demonstrated with hyperparameters selected on the test problem itself.

free parameters (5)
  • δd (decremental factor) = 0.5
    Decremental factor, 'determined from the experiments' for the PINN problem (Section 5.1).
  • δi (incremental factor) = 0.1
    Incremental factor, 'determined from the experiments' (Section 5.1).
  • δo (stagnation factor) = 1
    Stagnation factor, default value (Section 5.1).
  • initial learning rate (MNIST) = 0.01
    Specified in Section 5.5 as the chosen rate for MNIST experiments.
  • initial learning rate (CIFAR-10) = 0.005
    Specified in Section 5.5 for CIFAR-10 experiments.
assumptions (2)
  • domain assumption The normalized loss slope ΔLj = (L(B) − L(1))/L̄ within an epoch is a reliable signal for increasing or decreasing the learning rate.
    Introduced in Section 3.1 and formalized in Section 3.2, without proof or independent validation.
  • ad hoc to paper The update rule α_{j+1} = α_j − 10^{floor(log10 α_j)} δ_case ΔL_j will keep training stable and convergent across the tested problems.
    The rule is a heuristic; no convergence analysis is provided. The behavior depends on the choice of δd, δo, δi.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Improving Neural Network Training using Dynamic Learning Rate Schedule for PINNs and Image Classification." pith.science (2026). https://pith.science/paper/FMEHCPRU

@misc{pith2026250721749,
  author       = {Pith},
  title        = {Pith review of: Improving Neural Network Training using Dynamic Learning Rate Schedule for PINNs and Image Classification},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FMEHCPRU}},
  note         = {Machine review of arXiv:2507.21749}
}
read the original abstract

Training neural networks can be challenging, especially as the complexity of the problem increases. Despite using wider or deeper networks, training them can be a tedious process, especially if a wrong choice of the hyperparameter is made. The learning rate is one of such crucial hyperparameters, which is usually kept static during the training process. Learning dynamics in complex systems often requires a more adaptive approach to the learning rate. This adaptability becomes crucial to effectively navigate varying gradients and optimize the learning process during the training process. In this paper, a dynamic learning rate scheduler (DLRS) algorithm is presented that adapts the learning rate based on the loss values calculated during the training process. Experiments are conducted on problems related to physics-informed neural networks (PINNs) and image classification using multilayer perceptrons and convolutional neural networks, respectively. The results demonstrate that the proposed DLRS accelerates training and improves stability.

Figures

Figures reproduced from arXiv: 2507.21749 by the authors.

Figure 1
Figure 1. Neural network architecture used for PINNs. [PITH_FULL_IMAGE:figures/full_fig_p019_1.png] view at source ↗
Figure 2
Figure 2. Results of training on PINNs. Top row shows the training loss: [PITH_FULL_IMAGE:figures/full_fig_p021_2.png] view at source ↗
Figure 3
Figure 3. MNIST dataset sample [PITH_FULL_IMAGE:figures/full_fig_p023_3.png] view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Neural network architecture used for training MNIST dataset. [PITH_FULL_IMAGE:figures/full_fig_p023_4.png]
Figure 5
Figure 5. Figure 5: CNN training for 10 epochs on the MNIST dataset for different batches: (a, e) - [PITH_FULL_IMAGE:figures/full_fig_p024_5.png]
Figure 6
Figure 6. Figure 6: CIFAR-10 dataset sample. 23 [PITH_FULL_IMAGE:figures/full_fig_p024_6.png]
Figure 7
Figure 7. Figure 7: Neural network architecture used for training CIFAR-10 dataset. [PITH_FULL_IMAGE:figures/full_fig_p025_7.png]
Figure 8
Figure 8. Figure 8: CNN training for 100 epochs on the CIFAR-10 dataset for different batches: (a, [PITH_FULL_IMAGE:figures/full_fig_p028_8.png]
Figure 9
Figure 9. Figure 9: Comparison of DLRS against Adacomp on the MNIST dataset for 10 epochs at [PITH_FULL_IMAGE:figures/full_fig_p031_9.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

52 extracted references · 33 canonical work pages

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION fin.entry bracket.state close.brackets = "]" * 'skip if add.period write newline FUNCTION new.block output.state before.all = 'skip after.block 'output.state := if FUNCTION new.sentence output.state after.block = 'skip output.state before.all = 'skip after.sentence 'output.state := if if FUNCTION not...

    work page

  2. [2]

    Amari, S. (1993). Backpropagation and stochastic gradient descent method. Neurocomputing , 5(4-5), 185--196. https://doi.org/10.1016/0925-2312(93)90006-O

    work page doi:10.1016/0925-2312(93)90006-o 1993

  3. [3]

    Anthony, T., Tian, Z., & Barber, D. (2017). Thinking fast and slow with deep learning and tree search. Advances in neural information processing systems , 30

    work page 2017

  4. [4]

    W., & Zhao, S

    Bao, G., Wei, G. W., & Zhao, S. (2004). Numerical solution of the H elmholtz equation with high wavenumbers. International Journal for Numerical Methods in Engineering , 59(3), 389--408. https://doi.org/10.1002/nme.883

    work page doi:10.1002/nme.883 2004

  5. [5]

    Behera, L., Kumar, S., & Patnaik, A. (2006). On adaptive learning rate that guarantees convergence in feedforward networks. IEEE Transactions on Neural Networks , 17(5), 1116--1125. https://doi.org/10.1109/TNN.2006.878121

    work page arXiv 2006

  6. [6]

    Bishop, C. M. & Nasrabadi, N. M. (2006). Pattern recognition and machine learning , volume 4. Springer

    work page 2006

  7. [7]

    P., Selman, B., & Weinberger, K

    Bjorck, N., Gomes, C. P., Selman, B., & Weinberger, K. Q. (2018). Understanding batch normalization. Advances in Neural Information Processing Systems (NeurIPS 2018) , volume 31. https://proceedings.neurips.cc/paper_files/paper/2018/file/36072923bfc3cf47745d704feb489480-Paper.pdf

    work page arXiv 2018

  8. [8]

    Bottou, L. (2010). Large-scale machine learning with stochastic gradient descent. Proceedings of COMPSTAT'2010: 19th International Conference on Computational StatisticsParis France, August 22-27, 2010 Keynote, Invited and Contributed Papers , 177--186

    work page 2010

Show all 52 references
  1. [9]

    Chen, S., Zhang, C., & Mu, H. (2024). An adaptive learning rate deep learning optimizer using long and short-term gradients based on G--L fractional-order derivative. Neural Processing Letters , 56(2), 106. https://doi.org/10.1007/s11063-024-11571-7

    2024 doi

  2. [10]

    Chen, T., Li, M., Li, Y., Lin, M., Wang, N., Wang, M., Xiao, T., Xu, B., Zhang, C., & Zhang, Z. (2015). Mxnet: A flexible and efficient machine learning library for heterogeneous distributed systems. arXiv preprint arXiv:1512.01274

    2015 arXiv

  3. [11]

    Z., & Hong, M

    Chen, X., Wu, S. Z., & Hong, M. (2020). Understanding gradient clipping in private SGD : A geometric perspective. Advances in Neural Information Processing Systems (NeurIPS 2020) , volume 33, 13773--13782. https://proceedings.neurips.cc/paper_files/paper/2020/file/9ecff5455677...

    2020

  4. [12]

    Chetlur, S., Woolley, C., Vandermersch, P., Cohen, J., Tran, J., Catanzaro, B., & Shelhamer, E. (2014). cudnn: Efficient primitives for deep learning. arXiv preprint arXiv:1410.0759

    2014 arXiv

  5. [13]

    Deng, L. (2012). The MNIST database of handwritten digit images for machine learning research [ B est of the web]. IEEE Signal Processing Magazine , 29(6), 141--142. https://doi.org/10.1109/MSP.2012.2211477

    2012

  6. [14]

    Gale, T., Elsen, E., & Hooker, S. (2019). The state of sparsity in deep neural networks. arXiv preprint arXiv:1902.09574

    2019 arXiv

  7. [15]

    W., & Gonzalez, J

    Golmant, N., Vemuri, N., Yao, Z., Feinberg, V., Gholami, A., Rothauge, K., Mahoney, M. W., & Gonzalez, J. (2018). On the computational inefficiency of large batch sizes for stochastic gradient descent. arXiv:1811.12941 . https://doi.org/10.48550/arXiv.1811.12941

    2018 doi arXiv

  8. [16]

    Goodfellow, I. (2016). Deep learning

    2016

  9. [17]

    Han, S., Mao, H., & Dally, W. J. (2015). Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding. arXiv preprint arXiv:1510.00149

    2015 arXiv

  10. [18]

    He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. Proceedings of the IEEE conference on computer vision and pattern recognition , 770--778

    2016

  11. [19]

    Kingma, D. P. & Ba, J. (2014). Adam: A method for stochastic optimization. arXiv:1412.6980 . https://doi.org/10.48550/arXiv.1412.6980

    2014 doi arXiv

  12. [20]

    & Hinton, G

    Krizhevsky, A. & Hinton, G. (2009). Learning multiple layers of features from tiny images . University of Toronto, Canada. http://www.cs.utoronto.ca/ kriz/learning-features-2009-TR.pdf

    2009

  13. [21]

    E., Likas, A., & Fotiadis, D

    Lagaris, I. E., Likas, A., & Fotiadis, D. I. (1998). Artificial neural networks for solving ordinary and partial differential equations. IEEE Transactions on Neural Networks , 9(5), 987--1000. https://doi.org/10.1109/72.712178

    1998 doi

  14. [22]

    LeCun, Y., Bottou, L., Bengio, Y., & Haffner, P. (2002). Gradient-based learning applied to document recognition. Proceedings of the IEEE , 86(11), 2278--2324

    2002

  15. [23]

    Li, Y., Fu, Y., Li, H., & Zhang, S. (2009). The improved training algorithm of back propagation neural network with self-adaptive learning rate. 2009 International Conference on Computational Intelligence and Natural Computing , volume 1, 73--76. https://doi.org/10.1109/CINC.2009.111

    2009 doi

  16. [24]

    Li, Y., Ren, X., Zhao, F., & Yang, S. (2021). A zeroth-order adaptive learning rate method to reduce cost of hyperparameter tuning for deep learning. Applied Sciences , 11(21), 10184. https://doi.org/10.3390/app112110184

    2021 doi

  17. [25]

    & Arora, S

    Li, Z. & Arora, S. (2019). An exponential learning rate schedule for deep learning. arXiv preprint arXiv:1910.07454

    2019 arXiv

  18. [26]

    Liu, Y., Zhang, M., Zhong, Z., Zeng, X., & Long, X. (2021). A comparative study of recently deep learning optimizers. International Conference on Algorithms, High Performance Computing, and Artificial Intelligence (AHPCAI 2021) , volume 12156, 101--109. https://doi.org/10.1117...

    2021 doi

  19. [27]

    & Hutter, F

    Loshchilov, I. & Hutter, F. (2016). Sgdr: Stochastic gradient descent with warm restarts. arXiv preprint arXiv:1608.03983

    2016 arXiv

  20. [28]

    D., Vrahatis, M

    Magoulas, G. D., Vrahatis, M. N., & Androulakis, G. S. (1999). Improving the convergence of the backpropagation algorithm using learning rate adaptation methods. Neural Computation , 11(7), 1769--1796. https://doi.org/10.1162/089976699300016223

    1999 doi

  21. [29]

    Micikevicius, P., Narang, S., Alben, J., Diamos, G., Elsen, E., Garcia, D., Ginsburg, B., Houston, M., Kuchaiev, O., Venkatesh, G., et al. (2017). Mixed precision training. arXiv preprint arXiv:1710.03740

    2017 arXiv

  22. [30]

    Narayanan, D., Shoeybi, M., Casper, J., LeGresley, P., Patwary, M., Korthikanti, V., Vainbrand, D., Kashinkunti, P., Bernauer, J., Catanzaro, B., et al. (2021). Efficient large-scale language model training on gpu clusters using megatron-lm. Proceedings of the international co...

    2021

  23. [31]

    V., Bartakke, P

    Narkhede, M. V., Bartakke, P. P., & Sutaone, M. S. (2022). A review on weight initialization strategies for neural networks. Artificial Intelligence Review , 55(1), 291--322. https://doi.org/10.1007/s10462-021-10033-z

    2022 doi

  24. [32]

    Philipp, G., Song, D., & Carbonell, J. G. (2017). The exploding gradient problem demystified-definition, prevalence, impact, origin, tradeoffs, and solutions. arXiv:1712.05577 . https://doi.org/10.48550/arXiv.1712.05577

    2017 doi arXiv

  25. [33]

    Raissi, M., Perdikaris, P., & Karniadakis, G. E. (2019). Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational Physics , 378, 686--707. https://doi.org/1...

    2019 doi

  26. [34]

    Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., & Chen, L.-C. (2018). Mobilenetv2: Inverted residuals and linear bottlenecks. Proceedings of the IEEE conference on computer vision and pattern recognition , 4510--4520

    2018

  27. [35]

    & Zisserman, A

    Simonyan, K. & Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556

    2014 arXiv

  28. [36]

    Smith, L. N. (2018). A disciplined approach to neural network hyper-parameters: Part 1--learning rate, batch size, momentum, and weight decay. arXiv:1803.09820 . https://doi.org/10.48550/arXiv.1803.09820

    2018 doi arXiv

  29. [37]

    Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015). Going deeper with convolutions. Proceedings of the IEEE conference on computer vision and pattern recognition , 1--9

    2015

  30. [38]

    Takase, T., Oyama, S., & Kurihara, M. (2018). Effective neural network training with adaptive learning rate based on training loss. Neural Networks , 101, 68--78. https://doi.org/10.1016/j.neunet.2018.01.016

    2018 doi

  31. [39]

    Tieleman, T. (2012). Lecture 6.5-rmsprop: Divide the gradient by a running average of its recent magnitude. COURSERA: Neural networks for machine learning , 4(2), 26

    2012

  32. [40]

    & Ghosh, P

    Veerababu, D. & Ghosh, P. K. (2024). Neural network based approach for solving problems in plane wave duct acoustics. Journal of Sound and Vibration , 585, 118476. https://doi.org/10.1016/j.jsv.2024.118476

    2024

  33. [41]

    Wan, L., Zeiler, M., Zhang, S., Le Cun, Y., & Fergus, R. (2013). Regularization of neural networks using D rop C onnect. Proceedings of the 30th International Conference on Machine Learning , volume 28, 1058--1066. http://proceedings.mlr.press/v28/wan13.pdf

    2013

  34. [42]

    Weir, M. K. (1991). A method for self-determination of adaptive learning rates in back propagation. Neural Networks , 4(3), 371--379. https://doi.org/10.1016/0893-6080(91)90073-E

    1991 doi

  35. [43]

    Wilson, D. R. & Martinez, T. R. (2001). The need for small learning rates on large problems. IJCNN'01. International Joint Conference on Neural Networks. Proceedings (Cat. No. 01CH37222) , volume 1, 115--119. https://doi.org/10.1109/IJCNN.2001.939002

    2001

  36. [44]

    You, Y., Gitman, I., & Ginsburg, B. (2017). Large batch training of convolutional networks. arXiv preprint arXiv:1708.03888

    2017 arXiv

  37. [45]

    Yu, F., Wang, D., Shelhamer, E., & Darrell, T. (2018). Deep layer aggregation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , 2403--2412. https://openaccess.thecvf.com/content_cvpr_2018/papers/Yu_Deep_Layer_Aggregation_CVPR_2018_paper.pdf

    2018

  38. [46]

    & Zhu, H

    Yu, T. & Zhu, H. (2020). Hyper-parameter optimization: A review of algorithms and applications. arXiv:2003.05689 . https://doi.org/https://doi.org/10.48550/arXiv.2003.05689

    2020 doi arXiv

  39. [47]

    & Sennrich, R

    Zhang, B. & Sennrich, R. (2019). Root mean square layer normalization. Advances in Neural Information Processing Systems , 32

    2019

  40. [48]

    Zheng, Q., Saponara, S., Tian, X., Yu, Z., Elhanashi, A., & Yu, R. (2024). A real-time constellation image classification method of wireless communication signals based on the lightweight network mobilevit. Cognitive Neurodynamics , 18(2), 659--671. https://doi.org/10.1007/s11...

    2024 doi

  41. [49]

    Zheng, Q., Tian, X., Yang, M., Han, S., Elhanashi, A., Saponara, S., & Kpalma, K. (2025a). Reconstruction error based implicit regularization method and its engineering application to lung cancer diagnosis. Engineering Applications of Artificial Intelligence , 139, 109439. htt...

    2025

  42. [50]

    Zheng, Q., Tian, X., Yu, Z., Ding, Y., Elhanashi, A., Saponara, S., & Kpalma, K. (2023). Mobilerat: a lightweight radio transformer method for automatic modulation classification in drone communication systems. Drones , 7(10), 596. https://doi.org/10.3390/drones7100596

    2023 doi

  43. [51]

    Zheng, Q., Tian, X., Yu, Z., Yang, M., Elhanashi, A., & Saponara, S. (2025b). Robust automatic modulation classification using asymmetric trilinear attention net with noisy activation function. Engineering Applications of Artificial Intelligence , 141, 109861. https://doi.org/...

    2025

  44. [52]

    Zheng, Q., Zhao, P., Zhang, D., & Wang, H. (2021). Mr-dcae: Manifold regularization-based deep convolutional autoencoder for unauthorized broadcasting identification. International Journal of Intelligent Systems , 36(12), 7204--7238. https://doi.org/10.1002/int.22586

    2021 doi

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.