Develops a matrix-free SRQ-based action for the AL volume operator that exactly preserves the kernel and supports large-scale Monte Carlo and spectral estimates without dense matrices.
Automatic differentiation in machine learning: a survey
9 Pith papers cite this work. Polarity classification is still indexing.
9
Pith papers citing it
abstract
Derivatives, mostly in the form of gradients and Hessians, are ubiquitous in machine learning. Automatic differentiation (AD), also called algorithmic differentiation or simply "autodiff", is a family of techniques similar to but more general than backpropagation for efficiently and accurately evaluating derivatives of numeric functions expressed as computer programs. AD is a small but established field with applications in areas including computational fluid dynamics, atmospheric sciences, and engineering design optimization. Until very recently, the fields of machine learning and AD have largely been unaware of each other and, in some cases, have independently discovered each other's results. Despite its relevance, general-purpose AD has been missing from the machine learning toolbox, a situation slowly changing with its ongoing adoption under the names "dynamic computational graphs" and "differentiable programming". We survey the intersection of AD and machine learning, cover applications where AD has direct relevance, and address the main implementation techniques. By precisely defining the main differentiation techniques and their interrelationships, we aim to bring clarity to the usage of the terms "autodiff", "automatic differentiation", and "symbolic differentiation" as these are encountered more and more in machine learning settings.
citation-role summary
background 2
method 1
citation-polarity summary
representative citing papers
ADELIA is the first AD-enabled INLA system that computes exact hyperparameter gradients via a structure-exploiting multi-GPU backward pass, delivering 4.2-7.9x per-gradient speedups and 5-8x better energy efficiency than finite differences on models with up to 1.9 million latent variables.
Mitigation strategy for exploding gradients at material boundaries in differentiable radiation transport enables stable, optimization-ready derivatives for detector design.
Cyclic inverse design on athermal disordered sphere packings produces an emergent marginally absorbing manifold that encodes return-point memory of the training range through gradient discontinuities.
LESnets integrates LES equations and the law of the wall into F-FNO to enable data-free, stable long-term predictions of wall-bounded turbulence at Re_tau up to 1000 on coarse grids, matching traditional LES accuracy at higher efficiency.
Gradient-based optimization of SUPER and FTPE pulse protocols via auto-differentiation and uniTEMPO yields higher preparation fidelities than resonant pi-pulses or standard two-photon excitation, with the advantage increasing at higher temperatures.
Physics-informed neural networks solve two-flavor neutrino oscillation equations in vacuum and matter with mean squared errors of order 10^{-3} to 10^{-4}, matching analytical results.
CMS measures the W boson mass as 80360.2 ± 9.9 MeV from 2016 data, consistent with the Standard Model prediction.
A discretized finite mixture model with ADVI identifies interpretable low- and high-risk clusters in Markov degradation hazard models for 280 industrial pumps, achieving 84x speedup over NUTS while enforcing stability constraints.
citing papers explorer
-
A matrix free action of the Ashtekar-Lewandowski volume operator of loop quantum gravity
Develops a matrix-free SRQ-based action for the AL volume operator that exactly preserves the kernel and supports large-scale Monte Carlo and spectral estimates without dense matrices.
-
ADELIA: Automatic Differentiation for Efficient Laplace Inference Approximations
ADELIA is the first AD-enabled INLA system that computes exact hyperparameter gradients via a structure-exploiting multi-GPU backward pass, delivering 4.2-7.9x per-gradient speedups and 5-8x better energy efficiency than finite differences on models with up to 1.9 million latent variables.
-
Exploring the Boundaries of Differentiable Radiation Transport and Detector Simulation
Mitigation strategy for exploding gradients at material boundaries in differentiable radiation transport enables stable, optimization-ready derivatives for detector design.
-
Large-eddy simulation nets (LESnets) based on physics-informed neural operator for wall-bounded turbulence
LESnets integrates LES equations and the law of the wall into F-FNO to enable data-free, stable long-term predictions of wall-bounded turbulence at Re_tau up to 1000 on coarse grids, matching traditional LES accuracy at higher efficiency.
-
Efficient optimisation of multi-parameter quantum control protocols for strongly-coupled systems
Gradient-based optimization of SUPER and FTPE pulse protocols via auto-differentiation and uniTEMPO yields higher preparation fidelities than resonant pi-pulses or standard two-photon excitation, with the advantage increasing at higher temperatures.
-
Physics-Informed Neural Networks for Solving Two-Flavor Neutrino Oscillations in Vacuum and Matter Environments for Atmospheric and Reactor Neutrinos
Physics-informed neural networks solve two-flavor neutrino oscillation equations in vacuum and matter with mean squared errors of order 10^{-3} to 10^{-4}, matching analytical results.
-
Heterogeneous Variational Inference for Markov Degradation Hazard Models: Discretized Mixture with Interpretable Clusters
A discretized finite mixture model with ADVI identifies interpretable low- and high-risk clusters in Markov degradation hazard models for 280 industrial pumps, achieving 84x speedup over NUTS while enforcing stability constraints.