Fractional-order Jacobian Matrix Differentiation and Its Application in Artificial Neural Networks

Chengli Zhou; Chunna Zhao; Junjie Ye; Kemeng Xiang; Xiaojun Zhou; Yaqun Huang

REVIEW 4 major objections 5 minor 37 references

Fractional-order Jacobian Matrix Differentiation and Its Application in Artificial Neural Networks

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper's central claim is that a new fractional-order Jacobian matrix differentiation, $\mathbf{J}^\alpha$, moves fractional-order gradient descent into neural-network hidden layers, and that on two time-series tasks the orders $\alpha…

desk verdict A real idea for fractional-order Autograd inside hidden layers, but Algorithm 1 computes the wrong gradient for any multi-output layer, and the experiments are too weak to rescue it. read the letter →

arxiv 2506.07408 v1 pith:QJQI32K2 submitted 2025-06-09 cs.LG cs.AI

Xiaojun zhou , Chunna Zhao , Yaqun Huang , Chengli Zhou , Junjie Ye , Kemeng Xiang This is my paper

classification cs.LGcs.AI MSC 26A3368T07

keywords fractional-ordergradientdescentJacobianmatrixdifferentiationautomaticneuralnetworkoptimizationFLinearmodulelearningrateschedulingregularization

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Fractional-order differentiation promises built-in learning-rate scheduling and regularization for neural-network training, but existing fractional-order gradient descent methods could apply it only at the output layer, because no fractional-order matrix differentiation fit into automatic differentiation. This paper proposes $\mathbf{J}^\alpha$, a fractional-order generalization of the integer-order Jacobian matrix, and uses it to define a matrix-based fractional-order chain rule that runs inside hidden layers, in the backward pass rather than in the optimizer step. The authors implement it as a fractional-order Linear (FLinear) module in the PyTorch framework and report on two real-world time-series regression tasks that $\alpha = 0.8$ and $0.9$ give better validation and test performance than integer order $\alpha = 1.0$, with unchanged GPU memory and only linear time overhead. The central claim is that this is a genuinely fractional gradient method, not a linear rescaling of the integer-order gradient, and that it deserves to be called an "excellent fractional-order gradient descent method" in deep learning.

What carries the argument

The load-bearing object is $\mathbf{J}^\alpha$, the fractional-order Jacobian matrix of Definition 1: the $pq \times mn$ matrix of $\alpha$-th partial derivatives of the vectorized matrix function $\operatorname{vec}(F(X))$ with respect to every entry of $\operatorname{vec}(X)$, computed by fractional symbolic differentiation rather than Grünwald-Letnikov, Riemann-Liouville, or Caputo numerical schemes. Its structural fact is the block decomposition of Eq. (11): for a linear layer the fractional Jacobian carries $n^2$ distinct $p \times m$ blocks, and taking the first diagonal block yields a differentiation matrix with the same shape as the integer-order $A^\top$, which is exactly what lets the chain rule of Eq. (17) and Algorithm 1 run inside the backward pass. The companion identity is Eq. (12), the power-function derivative formula with the sign term that preserves gradient direction.

What would settle it

On a fixed mini-batch of either dataset, compute the weight gradient twice at $\alpha = 0.9$ for an MLP whose first layer has more than one output: once with Algorithm 1 (a single differentiation matrix from the first column of $W$) and once with the full $\mathbf{J}^\alpha$ using one differentiation matrix per output column. If the two gradients are numerically close and yield the same loss trajectories, the single-block simplification is not the source of the reported gains; if they differ substantially, the paper's experiments characterize only the restricted variant, and the magnitude of the difference settles how much of the advantage survives a full-block implementation.

Watch

Extended reading notes

Core claim

The core discovery is that replacing integer-order derivatives by fractional symbolic derivatives inside the Jacobian matrix yields a workable matrix-valued chain rule for neural-network layers. For the linear layer $F(X) = AX$, the integer-order Jacobian is block-diagonal with identical blocks $A$, but the fractional-order Jacobian of Eq. (11) contains $n^2$ distinct blocks, because fractional derivatives of constants are not zero; when $\alpha = 1.0$ or $n = 1$, the two coincide. From the block matrix the paper extracts the first diagonal block $A_{11}$, whose size matches the integer-order differentiation matrix, so it can be multiplied by the back-propagated matrix $G$ under ordinary matrix multiplication. The companion formula, Definition 2, is $\partial^\alpha y/\partial w^\alpha = x/\Gamma(2-\alpha)\,|w|^{1-\alpha} + \operatorname{sign}(w)\,b/\Gamma(1-\alpha)\,|w|^{-\alpha}$ for $y = xw + b$ with $\alpha \in (0,1]$, and Algorithm 1 turns it into an implicit, all-matrix computation of the weight gradient. The paper argues that this gradient is neither a linear mapping of the integer-order gradient nor a variant of gradient descent, and that its extra $|w|^{-\alpha}$ term acts as a built-in penalty, which is why fractional orders can generalize better when the right $\alpha$ is chosen.

Load-bearing premise

Algorithm 1 builds one fractional differentiation matrix from the first column of the weight matrix and applies it to every column of the weight gradient, an identification that is exact only when $\alpha = 1.0$ or the weight matrix has a single column; for $\alpha < 1$ and multiple columns the fractional Jacobian's blocks genuinely differ, so the method's reported behaviour rests on treating them as interchangeable.

Editorial extensions

If this is right

Fractional-order gradients are computed on the computation graph during backpropagation, so replacing each Linear layer with FLinear turns an SGD optimizer into the FSGD optimizer without any change to Optimizer.step(); the same recipe yields FAdam and, if extended to other modules, a fractional-order Autograd ecosystem.
At $\alpha = 1.0$ the construction provably reduces to the integer-order gradient $X^\top G$, so the proposed method contains ordinary backpropagation as a special case and is a strict generalization of it.
The extra term proportional to $|w|^{-\alpha}$ acts like a built-in penalty function, so the paper predicts less overfitting on noisy datasets such as DJI without any explicit weight decay.
Because the paper measures convergence speed as only linearly affected by the order $\alpha$ when the fractional gradient is bounded, it concludes that the FSGD optimizer needs no separate convergence proof beyond the integer-order one.
On the two datasets, $\alpha = 0.8$ and $0.9$ improve test MSE and MAE over $\alpha = 1.0$, while $\alpha = 0.7$ degrades into gradient explosion, which the authors take as evidence that the practically useful orders lie close to but below 1.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The experiments exercise only the first diagonal block $A_{11}$, built from the first column of the weight matrix and applied to every column of the weight gradient; a faithful implementation using all $n^2$ blocks of $\mathbf{J}^\alpha$ (or one differentiation matrix per output column) would show whether the reported advantage survives the full fractional structure or belongs to this single-block
The regularization claim suggests a direct stress test the paper does not run: train identical MLPs with FLinear and with integer SGD on the same data with injected label noise or stronger overfitting, and measure how much test performance degrades; the built-in penalty mechanism predicts fractional orders should degrade less.
Because Eq. (12) contains $|w|^{-\alpha}$, fractional updates become large near zero weights; varying the weight-initialization scale across orders would reveal whether the reported $\alpha \approx 0.9$ window is robust to initialization, which the fixed-seed experiments do not probe.
The claimed linear time overhead is measured on one epoch of one dataset; scaling the test to deeper MLPs or to convolutional and attention layers would show whether the matrix-based solver remains cheap enough for the FCNN, FRNN, and FTransformer modules the paper sketches.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Dear colleague,

Here is my take on arXiv:2506.07408. The paper has a genuinely useful framing: instead of patching fractional gradients into Optimizer.step(), it tries to compute fractional-order derivatives inside Loss.backward() by extracting a block of the fractional-order Jacobian of a linear layer. The FLinear module and the discussion of why existing FGD methods often reduce to learning-rate or momentum tweaks are worth something. Eq. (12) is the standard Riemann-Liouville power-rule derivative with an added sign, and the paper is honest that the lower bound is an engineering default.

The problem is Algorithm 1. The stress-test note holds up. For a layer Y = XW + b, the fractional derivative of y_ik with respect to w_jk depends on w_jk and on the residual of the other terms in that column, so the correct differentiation matrix is different for each output column. Algorithm 1 builds one matrix from W[:,0] and applies it to every column. The paper itself proves in Appendix C that there are n^2 distinct blocks, so selecting A11 for all columns is not a simplification; it is a different update rule. It coincides with the claimed derivative only when alpha=1 or n=1. Every reported experiment uses n=48 and alpha<1, so Figures 4 and 5 are not evaluating the method described by Eqs. (12) and (15). That is a load-bearing flaw, not a cosmetic one.

The experiments would not rescue the paper anyway. Two datasets, one two-layer MLP with no activation, no repeated runs, no error bars, and alpha selected on the validation set. The paper says code and data are not provided. The convergence discussion is also thin: it asserts that bounded fractional gradients give the same convergence as integer-order and therefore no proof is needed. That may be plausible under a strong assumption, but it is asserted rather than argued.

What is genuinely new is the idea of using a fractional-order Jacobian block for hidden-layer differentiation and the design of a Linear module that plugs into PyTorch's backward. If the per-column construction were fixed and the experiments rerun with honest uncertainty, the core observation could be interesting. As submitted, the central algorithm does not compute what it claims.

My recommendation: desk reject. The idea might warrant a footnote or a future correction, but this manuscript is not ready for peer review.

Referee Report

4 major / 5 minor

Summary. The paper proposes fractional-order Jacobian matrix differentiation (J^α), defined via an elementwise fractional derivative applied to the entries of the Jacobian of a matrix function. It specializes the construction to a linear layer Y = XW + b, derives Eq. (12) as the fractional derivative of y = xw + b, and in Algorithm 1 computes a weight gradient using only the first diagonal block of J^α. The authors integrate this into PyTorch by replacing nn.Linear with FLinear, and they claim that fractional-order gradients computed in Loss.backward() yield an "excellent fractional-order gradient descent method" with built-in learning-rate scheduling and regularization. The claim is supported experimentally on DJI and ETTh1 with α ∈ {0.7, 0.8, 0.9, 1.0}, where α = 0.8 and 0.9 are reported to outperform the integer-order baseline.

Significance. If the construction were correct, the paper would be a meaningful step toward fractional-order autodifferentiation in hidden layers: it offers a matrix-based fractional chain rule, avoids explicit elementwise loops, and reduces to the integer-order case at α = 1. The manuscript also contains useful ingredients, including a taxonomy of existing FGD methods, explicit pseudocode for Algorithm 1, reproducible seeding, and time/memory comparisons. However, the central algorithmic claim is not supported, because Algorithm 1 applies one differentiation matrix built from the first column of W to every column of G, while the fractional derivative is column-dependent for α < 1. Consequently, the reported experiments do not evaluate the gradients that the paper defines, and the convergence claim is asserted rather than proved.

major comments (4)

[Section 3.3, Algorithm 1] The differentiation matrix is constructed from F = W[:,0] and the first bias component b[0] (lines 1, 5-8), and line 12 multiplies that single matrix by the entire upstream matrix G. According to Eq. (15), the fractional derivative ∂^α y_ik / ∂(w_jk)^α depends on w_jk and on the residual of output column k, namely (Σ_{l≠j} x_il w_lk + b_k). The correct fractional gradient therefore requires a column-specific differentiation matrix D_k for each output column k. The manuscript itself states in Section 3.1 that the n diagonal blocks of J^α are different, and Appendix C proves there are n^2 distinct blocks. Selecting A11 for all columns is not derived from Eqs. (15)-(17); Algorithm 1 returns D_1^T G rather than [D_1^T G[:,1], ..., D_n^T G[:,n]]. This coincides with the claimed gradient only when α = 1 or n = 1. Since the experiments set the prediction length to 48 and use α < 1, Figures 4 and 5 evaluate an update rule that is different from the fractional-order gradient defined by Eqs. (12), (15), and (17).
[Section 3.3, Convergence Analysis] The paragraph on convergence asserts that "their convergence characteristics are the same as those of their corresponding integer-order optimizers" once the objective is bounded and the fractional-order gradient is bounded, and that "there is no need to list separate formulas for proof." This is not a convergence analysis. Replacing the gradient with a different vector field changes both the fixed points and the descent directions, and standard SGD convergence results require specific conditions on the update direction that are not verified for the update defined by Eq. (12) and Algorithm 1. The term |w|^{-α} is singular at w = 0, and the sign function modifies the direction, so even the boundedness assumption on the fractional-order gradient is nontrivial and unproven. A central claim of the paper is that J^α is an excellent fractional-order gradient descent method; without a proof or a valid cited convergence theorem, that claim is unsupported.
[Section 3.2, Definition 2 and Appendix D] Eq. (12) is presented as the Riemann-Liouville derivative R_0 D_w^α of y = xw + b, but the derivation in Appendix D applies the power rule separately to xw and to the constant b. For w < 0, the Riemann-Liouville derivative with lower limit 0 is not given by |w|^{1-α} and |w|^{-α} with an inserted sign; the sign function changes the mathematical object. Since weights initialized randomly will cross zero during training, the formula used in Algorithm 1 is an ad hoc engineering modification rather than a consequence of fractional calculus. At minimum, the paper should state explicitly which definition is being used, including the treatment of negative w and of the lower bound, and prove Eq. (12) from that definition.
[Section 4.3 and Figure 5] The conclusion of superior performance is reached after selecting α on the validation set: the text after Figure 5 reports that α = 0.8 and 0.9 are the best among the four tried orders. This is hyperparameter tuning, not evidence of a general property of J^α. In addition, the experiments use one random seed per order and report no repeated trials or error bars, so the quantitative differences in Figure 5 cannot be separated from initialization noise. The paper should either restrict its claim to "for these datasets and after α selection, the tested fractional orders outperform α = 1.0" or provide statistical support for a broader claim.

minor comments (5)

[Figure 2] The figure label contains the typo "rooot node"; please correct it.
[Equations (8)-(9)] The notation ∂^α vecF(X) / ∂((vecX)^T)^α is not standard; since the entries in Eq. (9) are ordinary scalar partial derivatives, the definition should be stated entrywise to avoid ambiguity about the meaning of a fractional derivative of a vector with respect to a transposed vector.
[Section 4.3, item (4)] The text says α = 0.8 is the second-best order, but Figure 5 shows α = 0.8 and α = 0.9 as the best; please reconcile the narrative with the plotted metrics.
[Section 3.3, Time Complexity Analysis] The statement that FSGD has the same time complexity as SGD is an asymptotic statement, but Table 2 shows a wall-clock increase of roughly 2.6x for FSGD 0.9 over SGD; clarify that the complexity claim refers to order of growth, not constant factors.
[Algorithm 1, line 7] Line 7 constructs the bias term using only b[0]. Even aside from the column-selection issue, this does not implement PyTorch's broadcasting of a length-n bias vector for output columns greater than one.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the central fractional-order gradient construction is not reduced to its inputs by definition, by self-citation, or by fitted parameters.

full rationale

The paper's derivation chain is not circular. Definition 2 and Eq. (12) are grounded in the external RL power-rule formula from Podlubny (1998), and Appendix D derives the formula from that external result rather than from the paper's own conclusions. Definition 1 defines the fractional-order Jacobian matrix as a symbolic fractional derivative of the vectorized matrix function, and Eq. (11) expresses its block structure; the paper then explicitly chooses the first diagonal block for use in Algorithm 1. This choice is an implementation simplification, not a hidden equivalence: selecting A11 is acknowledged in the text and in Appendix C, which states that J^alpha has n^2 distinct blocks. The resulting update rule may be inconsistent with the full Jacobian for n>1 and alpha<1, but that is a correctness problem, not circularity. Likewise, the experimental conclusion that alpha in {0.8, 0.9} performs well is reached by comparing multiple orders on validation and test sets; this is standard hyperparameter comparison, not a fitted parameter being relabeled as a prediction. The self-citation to Zhou et al. (2025b) for the claim that optimal fractional orders are usually in 0.9<alpha<1.0 is corroborative and not load-bearing, because the paper's own Figures 4 and 5 independently display the order comparison. No equation is shown to reduce by construction to its input, and no load-bearing premise depends solely on a chain of the authors' own prior work. Therefore the appropriate circularity score is 0, with the caveat that the validity of Algorithm 1 as a fractional derivative for multi-column weight matrices is a substantive technical concern outside the circularity analysis.

Assumptions & free parameters 2 free parameters · 6 assumptions · 0 invented entities

The central derivation rests on the RL power rule, several domain assumptions about the sign and lower bound, and one ad hoc assumption that the first Jacobian block represents all weight columns. No invented entities are introduced. The free parameters are alpha and the arbitrary lower bound 0; both are chosen by hand.

free parameters (2)

fractional order alpha = 0.9 (chosen from {0.7, 0.8, 0.9, 1.0})
Alpha is a tunable hyperparameter, not measured; the final comparison selects alpha=0.9 as best, and the paper notes previous reports that the optimum lies in (0.9, 1.0).
lower bound 0 in RL derivative = 0
The lower bound is fixed to 0 by "statistical induction" rather than derived from initialization; it affects all gradients through |w|^{1-alpha} and |w|^{-alpha}.

assumptions (6)

standard math Riemann-Liouville derivative of w^p with lower bound 0 equals Gamma(p+1)/Gamma(p+1-alpha)|w|^{p-alpha}, applied symbolically to linear layers.
Eq. (12) and Appendix D rely on this power rule; valid for scalar power functions, but the extension to matrix weights via absolute values and sign is not standard.
standard math Fractional differentiation is a linear operator, so D^alpha(x w + b) = x D^alpha w + b D^alpha 1.
Appendix D depends on linearity of the RL operator, which holds for the RL definition.
ad hoc to paper The sign function preserves or corrects gradient direction for fractional-power terms.
Introduced in Eq. (12) to avoid complex numbers; no theorem ensures descent direction, and the first term omits a sign for negative w.
ad hoc to paper The first diagonal block A11 of J^alpha can be used to compute the gradient for all columns of the weight matrix.
Algorithm 1 and Section 3.3 assume this; invalid generally since fractional blocks differ per column.
domain assumption Convergence of FSGD is the same as SGD whenever the objective and fractional gradient are bounded; no separate convergence proof is needed.
Section 3.3 Convergence Analysis; unproved and inconsistent with the non-smooth trajectories in Section 4.3.
standard math The fractional derivative of a constant bias term is b |w|^{-alpha}/Gamma(1-alpha), which acts as a built-in regularizer.
Appendix B follows from the RL power rule, but the regularization interpretation is asserted, not tested against L1 or L2.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Fractional-order Jacobian Matrix Differentiation and Its Application in Artificial Neural Networks." pith.science (2026). https://pith.science/paper/QJQI32K2

@misc{pith2026250607408,
  author       = {Pith},
  title        = {Pith review of: Fractional-order Jacobian Matrix Differentiation and Its Application in Artificial Neural Networks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QJQI32K2}},
  note         = {Machine review of arXiv:2506.07408}
}

abstract

Fractional-order differentiation has many characteristics different from integer-order differentiation. These characteristics can be applied to the optimization algorithms of artificial neural networks to obtain better results. However, due to insufficient theoretical research, at present, there is no fractional-order matrix differentiation method that is perfectly compatible with automatic differentiation (Autograd) technology. Therefore, we propose a fractional-order matrix differentiation calculation method. This method is introduced by the definition of the integer-order Jacobian matrix. We denote it as fractional-order Jacobian matrix differentiation (${{\bf{J}}^\alpha }$). Through ${{\bf{J}}^\alpha }$, we can carry out the matrix-based fractional-order chain rule. Based on the Linear module and the fractional-order differentiation, we design the fractional-order Autograd technology to enable the use of fractional-order differentiation in hidden layers, thereby enhancing the practicality of fractional-order differentiation in deep learning. In the experiment, according to the PyTorch framework, we design fractional-order Linear (FLinear) and replace nn.Linear in the multilayer perceptron with FLinear. Through the qualitative analysis of the training set and validation set $Loss$, the quantitative analysis of the test set indicators, and the analysis of time consumption and GPU memory usage during model training, we verify the superior performance of ${{\bf{J}}^\alpha }$ and prove that it is an excellent fractional-order gradient descent method in the field of deep learning.

Figures

Figures reproduced from arXiv: 2506.07408 by the authors.

**Figure 2.** Computational graph of ANN and 4 cases of FGD in back-propagation. [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗

**Figure 3.** The overall structure of the paper [PITH_FULL_IMAGE:figures/full_fig_p011_3.png] view at source ↗

Figures from the paper (3 more)

**Figure 4.** Figure 4: The convergence curves of DJI and ETTh1 on the training and validation sets. [PITH_FULL_IMAGE:figures/full_fig_p013_4.png]

**Figure 5.** Figure 5: The test set metrics on different α. Overall in [PITH_FULL_IMAGE:figures/full_fig_p014_5.png]

**Figure 4.** Figure 4: Analysis of Time Consumption and GPU Memory Usage: To analyze the time consumption and GPU memory usage of the FSGD optimizer (in this paper, it refers to SGD + FLinear) and the integer-order optimizer, we record the running time and GPU memory at different orders for …

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

37 extracted references · 31 canonical work pages

[1]

Robust and discriminative image representation: Fractional-order jacobi-fourier moments

Hongying Yang, Shuren Qi, Jialin Tian, Panpan Niu, and Xiangyang Wang. Robust and discriminative image representation: Fractional-order jacobi-fourier moments. Pattern Recognition, 115: 0 107898, 2021

work page 2021
[2]

Fractional poisson enhancement model for text detection and recognition in video frames

Sangheeta Roy, Palaiahnakote Shivakumara, Hamid A Jalab, Rabha W Ibrahim, Umapada Pal, and Tong Lu. Fractional poisson enhancement model for text detection and recognition in video frames. Pattern Recognition, 52: 0 433--447, 2016

work page 2016
[3]

Image analysis by fractional-order weighted spherical bessel-fourier moments

Tengfei Yang, Zhiquan Liu, Jingjing Guo, Yong Yu, Fang Ren, and Teng Wang. Image analysis by fractional-order weighted spherical bessel-fourier moments. Pattern Recognition, 157: 0 110872, 2025

work page 2025
[4]

Study on fast speed fractional order gradient descent method and its application in neural networks

Yong Wang, Yuli He, and Zhiguang Zhu. Study on fast speed fractional order gradient descent method and its application in neural networks. Neurocomputing, 489: 0 366--376, 2022

work page 2022
[5]

Fractional stochastic gradient descent for recommender systems

Zeshan Aslam Khan, Naveed Ishtiaq Chaudhary, and Syed Zubair. Fractional stochastic gradient descent for recommender systems. Electronic Markets, 29: 0 275--285, 2019

work page 2019
[6]

Study on fractional order gradient methods

Yuquan Chen, Qing Gao, Yiheng Wei, and Yong Wang. Study on fractional order gradient methods. Applied Mathematics and Computation, 314: 0 310--321, 2017

work page 2017
[7]

Fractional-order stochastic gradient descent method with momentum and energy for deep neural networks

Xingwen Zhou, Zhenghao You, Weiguo Sun, Dongdong Zhao, and Shi Yan. Fractional-order stochastic gradient descent method with momentum and energy for deep neural networks. Neural Networks, 181: 0 106810, 2025 a

work page 2025
[8]

Variable order fractional gradient descent method and its application in neural networks optimization

Weipu Lou, Wei Gao, Xianwei Han, and Yimin Zhang. Variable order fractional gradient descent method and its application in neural networks optimization. In 2022 34th Chinese Control and Decision Conference (CCDC), pages 109--114. IEEE, 2022

work page 2022

Show all 37 references

[9]

A deep learning optimizer based on gr \"u nwald--letnikov fractional order definition

Xiaojun Zhou, Chunna Zhao, and Yaqun Huang. A deep learning optimizer based on gr \"u nwald--letnikov fractional order definition. Mathematics, 11 0 (2): 0 316, 2023

2023
[10]

A fractional-order momentum optimization approach of deep neural networks

ZhongLiang Yu, Guanghui Sun, and Jianfeng Lv. A fractional-order momentum optimization approach of deep neural networks. Neural Computing and Applications, 34 0 (9): 0 7091--7111, 2022

2022
[11]

Convolutional neural networks based on fractional-order momentum for parameter training

Tao Kan, Zhe Gao, Chuang Yang, and Jing Jian. Convolutional neural networks based on fractional-order momentum for parameter training. Neurocomputing, 449: 0 85--99, 2021

2021
[12]

A fractional gradient descent algorithm robust to the initial weights of multilayer perceptron

Xuetao Xie, Yi-Fei Pu, and Jian Wang. A fractional gradient descent algorithm robust to the initial weights of multilayer perceptron. Neural Networks, 158: 0 154--170, 2023

2023
[13]

A comprehensive survey of fractional gradient descent methods and their convergence analysis

Sroor M Elnady, Mohamed El-Beltagy, Ahmed G Radwan, and Mohammed E Fouda. A comprehensive survey of fractional gradient descent methods and their convergence analysis. Chaos, Solitons & Fractals, 194: 0 116154, 2025

2025
[14]

Improved fractional-order gradient descent method based on multilayer perceptron

Xiaojun Zhou, Chunna Zhao, Yaqun Huang, Chengli Zhou, and Junjie Ye. Improved fractional-order gradient descent method based on multilayer perceptron. Neural Networks, 183: 0 106970, 2025 b

2025
[15]

An adaptive fractional-order bp neural network based on extremal optimization for handwritten digits recognition

Min-Rong Chen, Bi-Peng Chen, Guo-Qiang Zeng, Kang-Di Lu, and Ping Chu. An adaptive fractional-order bp neural network based on extremal optimization for handwritten digits recognition. Neurocomputing, 391: 0 260--272, 2020

2020
[16]

Fractional steepest ascent method for tcu fault detection

Hongqiu Zhu, Zhiliang Wu, Chunhua Yang, Tao Peng, Zhiwen Chen, and Xiaoyue Yang. Fractional steepest ascent method for tcu fault detection. IFAC-PapersOnLine, 51 0 (24): 0 1336--1342, 2018

2018
[17]

Fractional-order convolutional neural networks with population extremal optimization

Bi-Peng Chen, Yun Chen, Guo-Qiang Zeng, and Qingshan She. Fractional-order convolutional neural networks with population extremal optimization. Neurocomputing, 477: 0 36--45, 2022

2022
[18]

Performance analysis of fractional learning algorithms

Abdul Wahab, Shujaat Khan, Imran Naseem, and Jong Chul Ye. Performance analysis of fractional learning algorithms. IEEE Transactions on Signal Processing, 70: 0 5164--5177, 2022

2022
[19]

Artificial neural networks: a practical review of applications involving fractional calculus

E Viera-Martin, JF G \'o mez-Aguilar, JE Sol \' s-P \'e rez, JA Hern \'a ndez-P \'e rez, and RF Escobar-Jim \'e nez. Artificial neural networks: a practical review of applications involving fractional calculus. The European Physical Journal Special Topics, 231 0 (10): 0 2059--...

2022
[20]

A survey of fractional calculus applications in artificial neural networks

Manisha Joshi, Savita Bhosale, and Vishwesh A Vyawahare. A survey of fractional calculus applications in artificial neural networks. Artificial Intelligence Review, pages 1--54, 2023

2023
[21]

An overview of gradient descent optimization algorithms

Sebastian Ruder. An overview of gradient descent optimization algorithms. arXiv preprint arXiv:1609.04747, 2016

2016 arXiv
[22]

On the momentum term in gradient descent learning algorithms

Ning Qian. On the momentum term in gradient descent learning algorithms. Neural networks, 12 0 (1): 0 145--151, 1999

1999
[23]

Nesterov

Yu. Nesterov. A method of solving a convex programming problem with convergence rate o (1/k2). In Soviet Mathematics Doklady, 1983

1983
[24]

Two problems with backpropagation and other steepest-descent learning procedures for networks

Richard S Sutton. Two problems with backpropagation and other steepest-descent learning procedures for networks. In Proceedings of the Annual Meeting of the Cognitive Science Society, volume 8, 1986

1986
[25]

Adaptive subgradient methods for online learning and stochastic optimization

John Duchi, Elad Hazan, and Yoram Singer. Adaptive subgradient methods for online learning and stochastic optimization. Journal of machine learning research, 12 0 (7), 2011

2011
[26]

Adadelta: an adaptive learning rate method

Matthew D Zeiler. Adadelta: an adaptive learning rate method. arXiv preprint arXiv:1212.5701, 2012

2012 arXiv
[27]

Lecture 6.5-rmsprop: Divide the gradient by a running average of its recent magnitude

Tijmen Tieleman, Geoffrey Hinton, et al. Lecture 6.5-rmsprop: Divide the gradient by a running average of its recent magnitude. COURSERA: Neural networks for machine learning, 4 0 (2): 0 26--31, 2012

2012
[28]

Adam: A method for stochastic optimization

Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014

2014 arXiv
[29]

Sgdr: Stochastic gradient descent with warm restarts

Ilya Loshchilov and Frank Hutter. Sgdr: Stochastic gradient descent with warm restarts. arXiv preprint arXiv:1608.03983, 2016

2016 arXiv
[30]

A closer look at deep learning heuristics: Learning rate restarts, warmup and distillation

Akhilesh Gotmare, Nitish Shirish Keskar, Caiming Xiong, and Richard Socher. A closer look at deep learning heuristics: Learning rate restarts, warmup and distillation. arXiv preprint arXiv:1810.13243, 2018

2018 arXiv
[31]

Accelerating gradient descent and adam via fractional gradients

Yeonjong Shin, J \'e r \^o me Darbon, and George Em Karniadakis. Accelerating gradient descent and adam via fractional gradients. Neural Networks, 161: 0 185--201, 2023

2023
[32]

Fractional-order gradient descent learning of bp neural networks with caputo derivative

Jian Wang, Yanqing Wen, Yida Gou, Zhenyun Ye, and Hua Chen. Fractional-order gradient descent learning of bp neural networks with caputo derivative. Neural networks, 89: 0 19--30, 2017

2017
[33]

Fractional gradient descent algorithms for systems with outliers: A matrix fractional derivative or a scalar fractional derivative

Yuan Cao and Shuai Su. Fractional gradient descent algorithms for systems with outliers: A matrix fractional derivative or a scalar fractional derivative. Chaos, Solitons & Fractals, 174: 0 113881, 2023

2023
[34]

Fractional light gradient boosting machine ensemble learning model: A non-causal fractional difference descent approach

Haixin Wu, Yaqian Mao, Jiacheng Weng, Yue Yu, and Jianhong Wang. Fractional light gradient boosting machine ensemble learning model: A non-causal fractional difference descent approach. Information Fusion, 118: 0 102947, 2025

2025
[35]

Online public opinion prediction based on rolling fractional grey model with new information priority

Shuli Yan, Qi Su, Zaiwu Gong, Xiangyan Zeng, and Enrique Herrera-Viedma. Online public opinion prediction based on rolling fractional grey model with new information priority. Information Fusion, 91: 0 277--298, 2023

2023
[36]

Fractional differential equations: an introduction to fractional derivatives, fractional differential equations, to methods of their solution and some of their applications

Igor Podlubny. Fractional differential equations: an introduction to fractional derivatives, fractional differential equations, to methods of their solution and some of their applications. Elsevier, 1998

1998
[37]

Fractional-order gradient approach for optimizing neural networks: A theoretical and empirical analysis

Priyanka Harjule, Rinki Sharma, and Rajesh Kumar. Fractional-order gradient approach for optimizing neural networks: A theoretical and empirical analysis. Chaos, Solitons & Fractals, 192: 0 116009, 2025

2025

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

Robust and discriminative image representation: Fractional-order jacobi-fourier moments

Hongying Yang, Shuren Qi, Jialin Tian, Panpan Niu, and Xiangyang Wang. Robust and discriminative image representation: Fractional-order jacobi-fourier moments. Pattern Recognition, 115: 0 107898, 2021

work page 2021

[2] [2]

Fractional poisson enhancement model for text detection and recognition in video frames

Sangheeta Roy, Palaiahnakote Shivakumara, Hamid A Jalab, Rabha W Ibrahim, Umapada Pal, and Tong Lu. Fractional poisson enhancement model for text detection and recognition in video frames. Pattern Recognition, 52: 0 433--447, 2016

work page 2016

[3] [3]

Image analysis by fractional-order weighted spherical bessel-fourier moments

Tengfei Yang, Zhiquan Liu, Jingjing Guo, Yong Yu, Fang Ren, and Teng Wang. Image analysis by fractional-order weighted spherical bessel-fourier moments. Pattern Recognition, 157: 0 110872, 2025

work page 2025

[4] [4]

Study on fast speed fractional order gradient descent method and its application in neural networks

Yong Wang, Yuli He, and Zhiguang Zhu. Study on fast speed fractional order gradient descent method and its application in neural networks. Neurocomputing, 489: 0 366--376, 2022

work page 2022

[5] [5]

Fractional stochastic gradient descent for recommender systems

Zeshan Aslam Khan, Naveed Ishtiaq Chaudhary, and Syed Zubair. Fractional stochastic gradient descent for recommender systems. Electronic Markets, 29: 0 275--285, 2019

work page 2019

[6] [6]

Study on fractional order gradient methods

Yuquan Chen, Qing Gao, Yiheng Wei, and Yong Wang. Study on fractional order gradient methods. Applied Mathematics and Computation, 314: 0 310--321, 2017

work page 2017

[7] [7]

Fractional-order stochastic gradient descent method with momentum and energy for deep neural networks

Xingwen Zhou, Zhenghao You, Weiguo Sun, Dongdong Zhao, and Shi Yan. Fractional-order stochastic gradient descent method with momentum and energy for deep neural networks. Neural Networks, 181: 0 106810, 2025 a

work page 2025

[8] [8]

Variable order fractional gradient descent method and its application in neural networks optimization

Weipu Lou, Wei Gao, Xianwei Han, and Yimin Zhang. Variable order fractional gradient descent method and its application in neural networks optimization. In 2022 34th Chinese Control and Decision Conference (CCDC), pages 109--114. IEEE, 2022

work page 2022