Develops a matrix-free SRQ-based action for the AL volume operator that exactly preserves the kernel and supports large-scale Monte Carlo and spectral estimates without dense matrices.
Automatic differentiation in machine learning: a survey
9 Pith papers cite this work. Polarity classification is still indexing.
9
Pith papers citing it
abstract
Derivatives, mostly in the form of gradients and Hessians, are ubiquitous in machine learning. Automatic differentiation (AD), also called algorithmic differentiation or simply "autodiff", is a family of techniques similar to but more general than backpropagation for efficiently and accurately evaluating derivatives of numeric functions expressed as computer programs. AD is a small but established field with applications in areas including computational fluid dynamics, atmospheric sciences, and engineering design optimization. Until very recently, the fields of machine learning and AD have largely been unaware of each other and, in some cases, have independently discovered each other's results. Despite its relevance, general-purpose AD has been missing from the machine learning toolbox, a situation slowly changing with its ongoing adoption under the names "dynamic computational graphs" and "differentiable programming". We survey the intersection of AD and machine learning, cover applications where AD has direct relevance, and address the main implementation techniques. By precisely defining the main differentiation techniques and their interrelationships, we aim to bring clarity to the usage of the terms "autodiff", "automatic differentiation", and "symbolic differentiation" as these are encountered more and more in machine learning settings.
citation-role summary
background 2
method 1
citation-polarity summary
representative citing papers
ADELIA is the first AD-enabled INLA system that computes exact hyperparameter gradients via a structure-exploiting multi-GPU backward pass, delivering 4.2-7.9x per-gradient speedups and 5-8x better energy efficiency than finite differences on models with up to 1.9 million latent variables.
Mitigation strategy for exploding gradients at material boundaries in differentiable radiation transport enables stable, optimization-ready derivatives for detector design.
Cyclic inverse design on athermal disordered sphere packings produces an emergent marginally absorbing manifold that encodes return-point memory of the training range through gradient discontinuities.
LESnets integrates LES equations and the law of the wall into F-FNO to enable data-free, stable long-term predictions of wall-bounded turbulence at Re_tau up to 1000 on coarse grids, matching traditional LES accuracy at higher efficiency.
Gradient-based optimization of SUPER and FTPE pulse protocols via auto-differentiation and uniTEMPO yields higher preparation fidelities than resonant pi-pulses or standard two-photon excitation, with the advantage increasing at higher temperatures.
Physics-informed neural networks solve two-flavor neutrino oscillation equations in vacuum and matter with mean squared errors of order 10^{-3} to 10^{-4}, matching analytical results.
CMS measures the W boson mass as 80360.2 ± 9.9 MeV from 2016 data, consistent with the Standard Model prediction.
A discretized finite mixture model with ADVI identifies interpretable low- and high-risk clusters in Markov degradation hazard models for 280 industrial pumps, achieving 84x speedup over NUTS while enforcing stability constraints.
citing papers explorer
-
High-precision measurement of the W boson mass with the CMS experiment
CMS measures the W boson mass as 80360.2 ± 9.9 MeV from 2016 data, consistent with the Standard Model prediction.