REVIEW 9 cited by
The Elements of Differentiable Programming
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Artificial intelligence has recently experienced remarkable advances, fueled by large models, vast datasets, accelerated hardware, and, last but not least, the transformative power of differentiable programming. This new programming paradigm enables end-to-end differentiation of complex computer programs (including those with control flows and data structures), making gradient-based optimization of program parameters possible. As an emerging paradigm, differentiable programming builds upon several areas of computer science and applied mathematics, including automatic differentiation, graphical models, optimization and statistics. This book presents a comprehensive review of the fundamental concepts useful for differentiable programming. We adopt two main perspectives, that of optimization and that of probability, with clear analogies between the two. Differentiable programming is not merely the differentiation of programs, but also the thoughtful design of programs intended for differentiation. By making programs differentiable, we inherently introduce probability distributions over their execution, providing a means to quantify the uncertainty associated with program outputs.
Forward citations
Cited by 9 Pith papers
-
Co-design of ground-based gravitational wave detector networks
A proof-of-concept ML pipeline co-designs two-site gravitational-wave detector networks, producing three example configurations with 15–44 km arms and BNS ranges up to 1160 Mpc.
-
Alleviating prior dependencies for DESI DR1 clustering fits through reparameterization
Jeffreys prior over EFTofLSS coefficients mitigates projection effects in DESI DR1 power spectrum multipole fits, recentering posteriors for late-time expansion parameters.
-
Wall Shear Stress Reconstruction from Concentration: Differentiable Physics and Physics-Informed Neural Networks
Differentiable physics recovers accurate wall shear stress from concentration observations across measurement scenarios where PINNs fail.
-
Autoregressive Language Models are Secretly Energy-Based Models: Insights into the Lookahead Capabilities of Next-Token Prediction
Under the chain rule of probability, autoregressive models and energy-based models are in exact bijection in function space, making the global optimum of teacher forcing equivalent to an energy-based model with implic...
-
Learning long range dependencies through time reversal symmetry breaking
RHEL computes backpropagation-equivalent gradients for Hamiltonian recurrent networks using finite differences of time-reversed, nudged trajectories, and matches BPTT accuracy on sequence tasks up to 50k steps.
-
Controlled Periodic Synchronization for Efficient Data-Parallel Training
Periodic gradient+parameter synchronization with SlowMo beats DDP by 2.44 pp (K=4) on a WAN while cutting average wall-clock time by 13.8%, but only under a fixed LR=0.1 protocol.
-
ADSEQ: A delay-aware autograd-compatible framework for spike-event delivery in SNNs
Delay-aware gradient-enabled spike-event queues are built in JAX and benchmarked on four accelerator platforms, showing queue choice strongly affects simulation and training performance.
-
Learned iterative networks: An operator learning perspective
Learned iterative reconstruction networks can be uniformly described as operator learning: the unrolled architecture fixes how to compute while the loss and data fix what to compute; for nonlinear inverse problems the...
-
Learning to Optimize by Differentiable Programming
A tutorial survey of differentiable-programming-based first-order optimization, with dual-based PyTorch case studies and no new results.
Discussion (0). Sign in to comment.