REVIEW 21 cited by
Efficient and Modular Implicit Differentiation
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Efficient and Modular Implicit Differentiation
abstract
Automatic differentiation (autodiff) has revolutionized machine learning. It allows to express complex computations by composing elementary ones in creative ways and removes the burden of computing their derivatives by hand. More recently, differentiation of optimization problem solutions has attracted widespread attention with applications such as optimization layers, and in bi-level problems such as hyper-parameter optimization and meta-learning. However, so far, implicit differentiation remained difficult to use for practitioners, as it often required case-by-case tedious mathematical derivations and implementations. In this paper, we propose automatic implicit differentiation, an efficient and modular approach for implicit differentiation of optimization problems. In our approach, the user defines directly in Python a function $F$ capturing the optimality conditions of the problem to be differentiated. Once this is done, we leverage autodiff of $F$ and the implicit function theorem to automatically differentiate the optimization problem. Our approach thus combines the benefits of implicit differentiation and autodiff. It is efficient as it can be added on top of any state-of-the-art solver and modular as the optimality condition specification is decoupled from the implicit differentiation mechanism. We show that seemingly simple principles allow to recover many existing implicit differentiation methods and create new ones easily. We demonstrate the ease of formulating and solving bi-level optimization problems using our framework. We also showcase an application to the sensitivity analysis of molecular dynamics.
Forward citations
Cited by 21 Pith papers
-
Random test functions, $H^{-1}$ norm equivalence, and stochastic variational physics-informed neural networks
H^{-1} norm equivalence to expected squared evaluations on domain-dependent random test functions enables SV-PINNs that recover accurate solutions to challenging second-order elliptic PDEs faster than standard PINNs.
-
Scaling Nonlinear Optimization: Many Problems One GPU
jaxipm is the first GPU-batched IPOPT solver in JAX using heterogeneous iteration fusion and iteration-level batching, delivering up to 32.85x higher throughput than standard IPOPT on quadrotor NMPC benchmarks.
-
Neural Parameter Calibration for Finite-State Mean Field Games
A differentiable neural framework for learning state- and time-dependent parameters of finite-state mean field games from population trajectories via implicit differentiation.
-
Regularized Large Neighborhood Search
RLNS regularizes LNS to perform block Gibbs sampling under entropy, interpolating between pseudolikelihood and exact MLE for differentiable combinatorial optimization.
-
Learning Approximate Solutions to Multiparametric Generalized Nash Equilibrium Problems
A learning approach trains neural networks to approximate solutions of multiparametric GNEPs using NI gap loss with value surrogates, achieving large speedups and providing new existence conditions for continuous selections.
-
Random test functions, $H^{-1}$ norm equivalence, and stochastic variational physics-informed neural networks
Proves H^{-1} norm equivalence to expectation over random test functions and introduces SV-PINNs that outperform standard PINNs on eight elliptic problems.
-
jaxdae: A JAX-native Differentiable Solver for Differential-Algebraic Equations in Coupled Multi-physics
jaxdae provides the first JAX-native differentiable DAE solver, using a frozen-grid BDF-2 replay adjoint for reverse-mode gradients and XLA-fused batched sweeps.
-
BOSS-CLAM: Utilizing a Constrained Linear Absorption Model to Infer Stellar Parameters from BOSS Spectra
A new generative pipeline, BOSS-CLAM, infers temperature, gravity, metallicity, and alpha-abundance for 1,708,214 SDSS-V BOSS spectra and releases a validated clean catalog of 915,514 stars.
-
Implicit differentiation of tensor network algorithms
PEPS energy gradients can be computed by implicit differentiation of characteristic equations for the contraction environment, avoiding unstable subroutine backpropagation and reducing asymptotic cost.
-
Restricted Dynamic Geometric Complexity: Path-Space Reduction and M\"obius--Jacobi Response
RDGC is the least affine-invariant length of an admissible metric path to a Hessian-relative condition target, with global Green and bordered Jacobi–KKT response laws on Hadamard path space.
-
Designing bistable nanostructures for target behavior
Introduces an inverse-design framework showing that energy barriers and binding-site separations in bistable nanostructures are readily programmable while transition-state location and full profile shape require more ...
-
Calibration of an Analog-to-Digital Conversion Nonlinearity in JWST/NIRISS
A data-driven model for periodic ADC integral nonlinearity in JWST/NIRISS is fitted to ramp residuals and applied to correct the ERS1366 WASP-39b transmission spectrum, reducing systematics at the 30ppm level.
-
Stochastic gravitational-wave background search using data from five pulsar timing arrays
Combined five-PTA dataset yields posterior on SGWB power-law amplitude and index consistent with nonzero signal but below 5-sigma significance, with reconstructed angular correlations matching the Hellings-Downs prediction.
-
General Purpose Inverse Design of Heterogeneous Finite-Sized Assemblies
A differentiable wrapper around an analytical equilibrium-yield calculation enables gradient-based design of dimers, temperature-switchable shells, and size-controlled polymers, validated by molecular dynamics.
-
linrax: A JAX Compatible, Simplex Method Linear Program Solver
linrax implements the simplex method in JAX, enabling JIT-compiled, differentiable LP solving with support for degenerate constraints.
-
Restricted Dynamic Geometric Complexity: Path-Space Reduction and M\"obius--Jacobi Response
Restricted dynamic geometric complexity measures the intrinsic affine-invariant path distance from an initial metric to a condition-number target when the metric family is structurally constrained, with exact LMI and ...
-
Accelerated Fourier SAT (AFSAT): Fully Realising a GPU-based Symmetric Pseudo-Boolean SAT Solver
AFSAT realizes FastFourierSAT as a production GPU solver for heterogeneous symmetric pseudo-Boolean SAT via JAX-compiled continuous local search, with tailored DFT for stability and near-linear multi-accelerator scaling.
-
Resource-Efficient Quantum Optimization via Higher-Order Encoding
HUBO encodings reduce qubit counts from n*m to n*ceil(log2 m) and cut CNOT counts by 89.6-100% in QAOA benchmarks on gate assignment, max k-colorable subgraph, and integer programming instances.
-
IQPopt: Fast optimization of instantaneous quantum polynomial circuits in JAX
IQPopt is a JAX-based software tool enabling classical optimization of IQP circuits with thousands of qubits via efficient simulation of Pauli-Z expectation values, plus a module for quantum generative model training.
-
The mass of TOI-1883 b: A low density super-Neptune in the ridge regime transiting an early-M dwarf
Mass of 13.7 Earth masses and density 0.4 g cm^{-3} measured for TOI-1883 b, a super-Neptune in the ridge regime around an early-M dwarf, with implications for disk migration and photoevaporation.
-
A Study of Parallel Continuous Local Search
Empirical study of parallel continuous local search for SAT finds redundant constraints can slow convergence, CLS works as a hybrid sub-solver, and search stabilizes quickly due to saddle-dense objectives.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.