{"id":"0c53f7d6-ad4b-4938-8c55-16937ba93fdf","arxiv_id":"2411.14855","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"A learned optimizer can predict fractional-order and step-size parameters, giving 99.2% convergence on Rosenbrock 2D when trained on that same function, but the underlying fractional-derivative approximation is unsupported.","lead":"This paper trains a small neural network to pick the fractional-derivative order and step size for a variant of gradient descent, and reports large convergence gains on a 2-D test function and a Lorenz-system parameter search. A generalist might read it as an incremental experiment in learned optimization, but the central approximation of the fractional derivative is not derived or validated.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Eq. 13 is not a Taylor expansion of any fractional derivative defined in §1.1, and no validation is provided; the central claim that a network predicts fractional order is unsupported.","rationale":"The reader's weakest assumption identifies Eq. 13 as the core unvalidated step, and my stress-test agrees. The equation is mathematically under-specified: no definition of the fractional derivative is given for the expansion, and simple monomial checks contradict the formula. For f(x)=x, the RL and Caputo derivatives are x^{1-α}/Γ(2-α), while Eq. 13 with ΔX=0 yields x/Γ(1-α). For f(x)=x^2, the exact fractional derivative is 2x^{2-α}/Γ(3-α), which is not reproducible by the claimed combination of f(X) and f'(X). The α→1 limit of the first term is not divergent (1/Γ(1-α) tends to 0 as α→1−), but the formula still lacks any derivation, error bound, or validation against a standard fractional derivative. Because Eq. 12's update is the entire mechanism connecting the predicted α to the optimization trajectory, the headline empirical results cannot be interpreted as evidence for fractional-order prediction without a direct numerical check. The reader's additional concerns about in-sample training and grid-search on the test loss are valid but secondary; the mathematical gap in Eq. 13 alone justifies rejection of the central claim as stated. A direct numerical comparison would settle the matter, and the paper could be revised either by deriving and validating the approximation or by reframing the contribution as a new non-fractional parametric update.","tokens_in":6079,"tokens_out":6700,"duration_ms":58139,"concrete_test":"Specify the intended evaluation point in Eq. 13 (which X and ΔX are used in Eq. 12), then compute the Riemann–Liouville or Grünwald–Letnikov derivative of f(x)=x^2 at a few points, for α in (0,1), and compare direction and magnitude with Eq. 13. Also re-run the Rosenbrock experiment with Eq. 13 replaced by a direct numerical evaluation of the GL derivative. If the angle between the Eq. 13 update and the numerically exact fractional gradient exceeds roughly 30 degrees, or if the performance disappears with the exact derivative, the reported results are not attributable to fractional-order prediction.","verdict_should_be":"REJECT","load_bearing_attack":"Eq. 13 is not a Taylor expansion of any fractional derivative defined in §1.1, and it is never validated. For ΔX=0, it gives D^α f(X) ≈ f(X)/Γ(1-α), but for f(x)=x the Riemann–Liouville and Caputo derivatives (Eqs. 4 and 6) are x^{1-α}/Γ(2-α); these differ by a function of X. Alternatively, if Eq. 13 is meant as an expansion in ΔX around X, no standard fractional derivative produces a leading term f(X)/Γ(1-α) that is independent of the lower limit and of X. Direct monomial checks (e.g., f(x)=x^2) contradict the formula. No derivation, remainder bound, or numerical comparison against a standard fractional derivative is given, so the network's predictions cannot be attributed to fractional order: the α-dependent term f(X)/Γ(1-α) acts as a scaled, state-dependent bias and the second term as a rescaled gradient. The gains in Table 1 may therefore come from a generic two-parameter update rather than fractional calculus, so the 99.2% and 71.8% results are not evidence for the abstract's claim.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a learned optimizer that predicts a fractional derivative order α and step size η, with the update X_{t+1}=X_t−ηD^α f(X_t) where D^α is approximated by the truncated expansion in Eq. (13). The method is tested on classical optimization functions, reporting a 99.2% convergence rate on Rosenbrock 2D when trained with supervision on the target function and 71.8% when trained without supervision, compared with 0.6% for gradient descent and 1.6% for Adam. A second set of experiments applies a fractional gradient flow discretization, Eq. (17), to Lorenz system parameter optimization and reports improved stability/convergence for backpropagation-through-time. The abstract concludes that a neural network can be trained to predict the order of the gradient effectively.","tokens_in":6275,"tokens_out":4133,"duration_ms":43424,"significance":"If the central claim were supported, the paper would make a useful contribution: learned selection of a fractional order in optimization is a novel idea, and the without-supervision Rosenbrock result (71.8% convergence vs. 0.6% for gradient descent) would be a striking demonstration that a small network can outperform standard optimizers on a held-out function. The paper also makes a reasonable effort to compare against strong baselines, including VeLO. However, the significance is entirely conditional on the validity of Eq. (13), the approximation that defines the optimizer's update. Because that approximation does not correspond to any standard fractional derivative and is not validated, the reported gains currently do not constitute evidence for the paper's abstract claim. The without-supervision experiment is a genuine attempt at independent evaluation, and the Lorenz experiment is an interesting probe, but both rely on the same unvalidated fractional-calculus foundation.","major_comments":[{"comment":"Equation (13) is not a Taylor expansion of the Riemann–Liouville or Caputo derivative defined in §1.1, and no derivation, error bound, or numerical validation is provided. For f(x)=x, both the RL and Caputo derivatives give x^{1−α}/Γ(2−α), whereas Eq. (13) evaluated at ΔX=0 gives x/Γ(1−α). For f(x)=x^2, the formula gives x^2/Γ(1−α)+2xΔX/Γ(α), whereas the actual fractional derivatives are x^{2−α}/Γ(3−α) plus terms involving the lower limit. The first term in Eq. (13) is a state-dependent scalar multiple of f(X) and does not match the X^{-α}-type prefactor produced by any standard fractional derivative. Since Eq. (12) uses this approximation for every update, the meta-learning results in Table 1 demonstrate only that a learned two-parameter update proportional to f(X) and ∇f(X) can optimize these functions; they do not show that the network predicts a meaningful fractional order.","section":"§2.1, Eq. (13)"},{"comment":"The headline 99.2% result is circular: row (8) is the model trained 'with supervision' on the target function and then tested on the same function. This is an in-sample fit, not evidence of generalization or of effective fractional-order prediction. The without-supervision row (9), which holds out the target function, is the relevant independent evaluation, and it should be the basis for any claim about predicting fractional order on unseen functions.","section":"Table 1 and §2.1"},{"comment":"The chaotic-system experiment is reported without enough detail to assess the claim that the FGF discretization makes TBTT 'perform the best.' The fractional order α in Eq. (16) appears to be a manually chosen free parameter, not predicted by the learned network, and no values of α or η, no error bars, and no number of independent runs are given. The comparison between NRES and TBTT also lacks a precise definition of the convergence metric. These omissions make it impossible to determine whether the improved stability comes from the fractional flow or from a differently tuned experiment.","section":"§2.2 and Fig. 2(b)"},{"comment":"The discretization in Eq. (17) is attributed to reference [3], but that reference concerns approximations of fractional-order transfer functions and does not appear to contain this optimization-oriented discretization. The authors should provide a derivation or a correct citation, and they should state the conditions under which the claimed O(1/t^α) convergence rate applies to the optimization setting of §2.2.","section":"Eq. (17) and reference [3]"}],"minor_comments":[{"comment":"The notation ΔX ∂f/∂X is ambiguous for vector-valued X; in a multivariate setting the second term should be written as ΔX·∇f or the equivalent scalar product.","section":"Eq. (13)"},{"comment":"The discussion of the fractional Jacobian would benefit from an explicit statement of how fractional derivatives are extended to multivariate functions and why the transformation T_{J^α_f} is called a 'linear approximation only when α=1.'","section":"§1.2, Eq. (10)"},{"comment":"The training loss Lθ = log(f(X_{t+1})) − log(f(X_t)) requires f to be positive on the sampled trajectories; the paper does not state that all classical functions are positive-valued, and the loss is undefined for negative function values.","section":"Eq. (14)"},{"comment":"The column header 'Truncated trajectory length' is not defined, and the reported values near 1000 are puzzling given the stated maximum of 100 optimization steps; the authors should clarify what this quantity measures.","section":"Table 1"},{"comment":"The paper does not provide training details for Fθ, such as network architecture, hidden sizes, batch size, optimizer hyperparameters, or the number of training steps, which makes the experiments difficult to reproduce.","section":"§3.1"},{"comment":"The subplots in Fig. 2 lack axis labels and colorbar units, and the fractional order/learning-rate sweep for the FGF method is not described in the text.","section":"Fig. 2"}],"recommendation":"reject","confidential_remarks":"The central problem is Eq. (13), which is not a valid expansion of any fractional derivative defined in the paper. Because this equation defines the update used in all meta-learning experiments, the main claim that the network predicts fractional order is unsupported. The circular with-supervision evaluation further weakens the headline result. These issues are not local presentation problems; fixing them would require replacing the update rule with a validated fractional-derivative approximation and rerunning all experiments. I would therefore recommend rejection, while noting that the general idea of learned fractional-order selection might be worth revisiting if a correct and validated discretization is used."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The new piece is genuine: the paper trains a small network to output both a fractional order alpha and a step size eta, and it compares against hand-designed adaptive fractional methods. The held-out experiment, with training on other functions and testing on Rosenbrock, is a reasonable design, and 71.8 percent convergence versus 0.6 percent for GD is striking if it holds. The chaotic-systems experiment applying a fractional gradient flow discretization to Lorenz parameter optimization is also sensible, and the authors are honest about toy-scale scope.\n\nThe problem is the engine. Eq. 13 is presented as a first-order truncated Taylor expansion, but it is not a Taylor expansion of any fractional derivative defined in Section 1.1. For f(x)=x, the Riemann-Liouville and Caputo derivatives at X are X^(1-alpha)/Gamma(2-alpha). Eq. 13 gives f(X)/Gamma(1-alpha) plus alpha DeltaX/Gamma(alpha). These differ. The leading term f(X)/Gamma(1-alpha) is a state-dependent bias that does not vanish as DeltaX goes to zero, which no local derivative expansion should produce. At alpha=1 the formula requires an infinite limit, and only after taking a limit does it reduce to a scaled gradient step. No derivation, no error bound, and no numerical check against a standard fractional derivative is provided.\n\nSo the headline numbers, 99.2 percent with supervision and 71.8 percent without, may simply reflect a generic two-parameter update with state-dependent scaling. The 99.2 percent result is in-sample by construction. The chaotic experiments grid-search alpha and eta on the test loss surface and report no error bars, making those comparisons hard to interpret.\n\nThe paper is not worthless. The idea of learning alpha is worth exploring, and the authors correctly identify a gap in the literature. But as written, the central claim is unsupported because the core approximation is mathematically inconsistent with the paper's own definitions. I would not send this to peer review in its current form; a serious referee would flag Eq. 13 immediately. If the authors can derive a proper expansion, validate it numerically, or use an established fractional derivative, a resubmission could be worth considering.","headline":"The learned prediction of alpha is a real idea, but Eq. 13 is not a Taylor expansion of any fractional derivative defined in the paper, so the reported gains do not support the claim about fractional order.","tokens_in":719,"tokens_out":645,"would_cite":false,"duration_ms":28039,"reading_group":"no","serious_thinker":"no","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Training a neural network to predict the fractional order of the gradient is possible, and on Rosenbrock 2D the resulting learned optimizer converges in 99.2% of runs.","keywords":["fractional calculus","fractional gradient descent","learned optimization","meta-learning","fractional gradient flow","chaotic systems","Rosenbrock function","backpropagation through time"],"falsifier":"Compute the true Riemann-Liouville or Caputo fractional derivative (Eq. 4 or Eq. 6, via numerical quadrature) or the Grünwald-Letnikov finite-difference form (Eq. 7) along the optimizer's trajectory, and compare its update direction with the Eq. (13) surrogate; if the directions disagree substantially, the 99.2% convergence is an artifact of the surrogate update. Alternatively, rerun the supervised experiment with $\\alpha$ clamped to 1 everywhere; if convergence remains near 99%, the learned fractional order itself is not doing the work.","tokens_in":5791,"feed_emoji":"🧮","tokens_out":6888,"duration_ms":58786,"temperature":0.7,"pith_summary":"The paper tries to establish that fractional-order gradient descent can be made practical by learning the order rather than tuning it by hand. It reports that a small neural network, fed the current state and gradient information, can output the fractional order $\\alpha$ and step size $\\eta$ so that the update $X_{t+1}=X_t-\\eta D^{\\alpha} f(X_t)$ converges on Rosenbrock 2D in 99.2% of 1000 random starts with supervision, and 71.8% when trained on other functions and tested on Rosenbrock, compared with at most 3.5% for standard optimizers. The same idea is applied to chaotic systems through a fractional gradient flow discretization, which the paper reports stabilizes backpropagation-through-time for Lorenz parameter optimization. If true, this would replace hand-designed update rules with a learned mechanism that behaves like a second-order method while only relying on first-order information.","feed_headline":"Network learns fractional gradient order: 99.2% convergence","feed_subtitle":"A tiny network predicting α and step size beats GD, Adam, and VeLO on Rosenbrock and stabilizes BPTT for Lorenz.","key_machinery":"The carrying object is a learned map to fractional order: $F_\\theta$ takes the current state, normalized gradient, gradient magnitude, and Fourier features and outputs $(\\alpha_t, \\eta_t)$, producing $X_{t+1}=X_t-\\eta_t D^{\\alpha_t} f(X_t)$. The fractional derivative is defined via the Riemann-Liouville, Caputo, and Grünwald-Letnikov formulations, but the paper approximates $D^{\\alpha}$ by a first-order truncated Taylor expansion with weights $\\frac{1}{\\Gamma(1-\\alpha)} f(X) + \\frac{1}{\\Gamma(\\alpha)} \\Delta X \\frac{\\partial f}{\\partial X}$ (Eq. 13). For chaotic systems the mechanism is the fractional gradient flow $d^{\\alpha} X/dt = -\\nabla f(X)$, discretized by the multi-step scheme of Eq. (17), which is what stabilizes backpropagation-through-time on the Lorenz parameter-optimization task. Fourier features are an auxiliary but important ingredient: the paper reports they were necessary for the network to learn high-frequency structure and generalize across functions.","core_discovery":"The central claim, stated on the paper's own terms, is that 'it is possible to train a neural network to predict the order of the gradient effectively.' Concretely, a learned optimizer $F_\\theta$ maps the current state, normalized gradient, gradient magnitude, and Fourier features to $(\\alpha, \\eta)$; the update is a fractional gradient step with $D^{\\alpha}$ approximated by the first-order truncated Taylor expression of Eq. (13). On Rosenbrock 2D this learned optimizer converges in 99.2% of runs with supervision and 71.8% without, whereas gradient descent converges in 0.6% and Adam in 1.6%. For chaotic systems, replacing the gradient update in backpropagation-through-time with the discretized fractional gradient flow of Eq. (17) makes the trajectory loss converge faster and more stably than the NRES gradient-estimator baseline. The paper frames these results as evidence that fractional calculus can be a viable component of learned optimization, while noting the current limitation to small-dimensional problems.","pith_inferences":["If Eq. (13) is a faithful local model, the same learning rule could be extended to per-dimension fractional orders, which the paper identifies as a major open problem; a natural next test is whether a Transformer or structured-state-space map predicts a vector of $\\alpha$ values for high-dimensional losses.","The FGF discretization's stabilizing effect on BPTT suggests a cheaper alternative to evolutionary-search gradient estimators in chaotic control problems; one could test whether the same scheme stabilizes BPTT on partially observable or stochastic chaotic systems.","It would be informative to check whether the learned order $\\alpha$ tracks curvature or conditioning (e.g., near the Rosenbrock valley), since a positive correlation would indicate the network is learning geometry rather than simply fitting a schedule."],"forward_implications":["Fractional-order optimizers need not be hand-tuned: a learned policy can predict both the order and the step size on the fly, and on Rosenbrock 2D the supervised policy reaches 99.2% convergence, a departure from the sub-4% rates of classical optimizers.","The unsupervised results (71.8%) indicate that a policy trained on other classical functions transfers to a held-out target, suggesting fractional-order prediction is a learnable skill rather than a memorized fit.","The fractional gradient flow discretization renders backpropagation-through-time competitive on the Lorenz parameter task, converging faster and more stably than the noise-reuse evolution-strategies gradient estimator NRES.","Fourier features are a necessary ingredient for the learned policy to handle high-frequency behavior in low dimensions, which constrains the architecture choices for a practical learned fractional optimizer."],"supporting_citations":[{"why":"Supplies the convergence-rate analysis for fractional gradient descent in the convex case that motivates the method.","marker":"[9]"},{"why":"AdamW is the optimizer used to train the neural network $F_\\theta$ against the log-loss objective.","marker":"[12]"},{"why":"Fourier features are the input representation the paper credits for making the learned policy generalize across classical functions.","marker":"[19]"},{"why":"Defines the Lorenz parameter-optimization task and supplies the NRES gradient estimator baseline that the fractional gradient flow update is compared against.","marker":"[8]"},{"why":"VeLO is the general-purpose learned optimizer used as a baseline in the Rosenbrock convergence comparison.","marker":"[14]"},{"why":"Provides the discretization scheme (Eq. 17) used to turn the fractional gradient flow into a practical update for the chaotic-system experiment.","marker":"[3]"}],"fun_headline_variants":["Neural net predicts fractional order for gradient descent","Learned fractional optimizer: 99.2% convergence on Rosenbrock","Fractional gradient order learned, stabilizes BPTT for Lorenz","Tiny network learns to set fractional gradient steps","Forecasting fractional order for faster optimization"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the first-order truncated Taylor expansion in Eq. (13), with its Gamma-function weights, really does approximate the fractional derivative in the optimization setting; the paper gives no derivation or error bound, and if the approximation is inaccurate the reported convergence gains do not establish that fractional order prediction works.","fun_headline_variants_meta":{"raw":{"variants":["Neural net predicts fractional order for gradient descent","Learned fractional optimizer: 99.2% convergence on Rosenbrock","Fractional gradient order learned, stabilizes BPTT for Lorenz","Tiny network learns to set fractional gradient steps","Forecasting fractional order for faster optimization"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000247,"raw_usage":{"total_tokens":1482,"prompt_tokens":825,"completion_tokens":657,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":441,"completion_tokens_details":{"reasoning_tokens":578}},"tokens_in":441,"tokens_out":657,"duration_ms":7272,"temperature":1.0,"reasoning_tokens":578,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T14:47:53.454220+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compute the true Riemann-Liouville or Caputo fractional derivative (Eq. 4 or Eq. 6, via numerical quadrature) or the Grünwald-Letnikov finite-difference form (Eq. 7) along the optimizer's trajectory, and compare its update direction with the Eq. (13) surrogate; if the directions disagree substantially, the 99.2% convergence is an artifact of the surrogate update. Alternatively, rerun the supervised experiment with $\\alpha$ clamped to 1 everywhere; if convergence remains near 99%, the learned fractional order itself is not doing the work.","supporting_citations":[{"cited_title":"Fractional differe ntial equation approach for convex optimization with convergence rate analysis","cited_arxiv_id":null,"evidence_quote":"Supplies the convergence-rate analysis for fractional gradient descent in the convex case that motivates the method."},{"cited_title":"Decoupled weight dec ay regularization","cited_arxiv_id":null,"evidence_quote":"AdamW is the optimizer used to train the neural network $F_\\theta$ against the log-loss objective."},{"cited_title":"Fourier features let net- works learn high frequency functions in low dimensional dom ains","cited_arxiv_id":null,"evidence_quote":"Fourier features are the input representation the paper credits for making the learned policy generalize across classical functions."},{"cited_title":"V ariance- reduced gradient estimation via noise-reuse in online evol ution strategies","cited_arxiv_id":null,"evidence_quote":"Defines the Lorenz parameter-optimization task and supplies the NRES gradient estimator baseline that the fractional gradient flow update is compared against."},{"cited_title":"Revisiting four approximation methods for fractional order transfer f unction implementations: Stabil- ity preservation, time and frequency response matching ana lyses","cited_arxiv_id":null,"evidence_quote":"Provides the discretization scheme (Eq. 17) used to turn the fractional gradient flow into a practical update for the chaotic-system experiment."}],"review_version":1}