Pith. sign in

REVIEW 2 major objections 4 minor 88 references

Gradient estimation of a probabilistic program reduces to ordinary inference on a factorized coupling, then standard automatic differentiation.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-10 16:56 UTC pith:SWVCSM5Z

load-bearing objection Clean formal reduction of gradient estimation to inference via typed couplings; recovers known estimators and delivers large, measured variance wins with proofs and artifact. the 2 major comments →

arxiv 2607.07840 v1 pith:SWVCSM5Z submitted 2026-07-08 cs.PL cs.LG

GradInf: Gradient Estimation as Probabilistic Inference

Gaurav Arya , Mathieu Huot , Moritz Schauer , Alexander K. Lew , Feras A. Saad This is my paper
classification cs.PL cs.LG
keywords probabilistic programminggradient estimationcouplingsautomatic differentiationsource-to-source transformationinformation-flow typingsequential Monte Carlovariable elimination
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Estimating the gradient of an expectation defined by a probabilistic program is hard: the integral is usually intractable, discrete randomness breaks ordinary automatic differentiation, and different estimators look like unrelated tricks. This paper claims the problem is not a new kind of differentiation but a probabilistic inference problem in disguise. First form a coupling that shares randomness between the original program and a slightly perturbed copy; then factor the coupled program so that one half (the primal) can be fixed by a random seed while the residual half is left free. Any sound algorithm that estimates the expected residual difference can then be differentiated by ordinary automatic differentiation, producing an unbiased gradient estimator. Because the reduction is modular, existing inference engines become gradient engines, known estimators reappear as particular choices of coupling plus inference, and new combinations (variable elimination or twisted sequential Monte Carlo on a carefully chosen coupling) cut time-adjusted variance by large factors on queuing, option-pricing and gene-transcription models.

Core claim

Any gradient of an expectation E[x] under a probabilistic program can be recovered, unbiasedly, by synthesizing a factorized coupling of the program, running an expectation-preserving inference algorithm on the residual choices, and differentiating the resulting estimator with ordinary automatic differentiation (Theorem 4).

What carries the argument

Gradient inference: the composition of two source-to-source transformations (a coupling C* that shares randomness and a partial-evaluation S that freezes the primal seed) followed by any sound inference macro and standard AD; soundness is proved by proof-relevant logical relations on a quasi-Borel denotational model.

Load-bearing premise

The random choices made by the inference algorithm itself must not depend on the finite perturbation that is later differentiated; otherwise the interchange of derivative and expectation fails.

What would settle it

On any of the three case studies, replace the residual sampler by one whose internal random seeds depend on the perturbation ε and check whether the Monte-Carlo mean of the resulting AD estimator still matches a high-accuracy finite-difference or score-function baseline within statistical error.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Any existing unbiased inference algorithm that estimates an ordinary expectation becomes, after the coupling-factorization transform, a ready-made unbiased gradient estimator.
  • Classic estimators (pathwise, SPA phantom, measure-valued derivatives, REINFORCE, DisARM, straight-through) are recovered simply by choosing different factorized couplings and different inference macros.
  • New combinations (CRN + variable elimination, maximal independent coupling + twisted SMC) deliver up to two orders of magnitude lower time-adjusted variance on the reported models.
  • Practitioners can design gradient estimators by composing off-the-shelf couplings and inference engines rather than inventing new differentiation rules.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same reduction immediately suggests higher-order gradient estimators: apply the construction to a first-order GradInf estimator itself.
  • Because the residual program is an ordinary probabilistic program without observe statements, any future improvement in exact or approximate inference (e.g., better SMC twists or GPU-accelerated VE) automatically improves the corresponding gradient estimator.
  • Information-flow typing that separates primal from residual could be reused to automate the discovery of low-variance control variates or of couplings that maximise correlation.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 4 minor

Summary. The paper introduces gradient inference: a reduction of the problem of estimating ∇_θ E_{x∼μ_θ}[x] for a probabilistic program to a probabilistic inference problem obtained by coupling and factorization, followed by ordinary automatic differentiation. It presents GradInf, a higher-order probabilistic programming system whose core is source-to-source transformations C, C*, E and S (coupling, factorized coupling, erasure, and partial probability evaluation) whose soundness is established by proof-relevant logical relations over quasi-Borel spaces (Theorems 1–3). Composition with any expectation-preserving inference macro I and any sound AD macro D yields a sound Monte Carlo gradient estimator (Theorem 4). The system recovers a range of existing estimators (pathwise/IPA, SPA phantoms, MVD, score, RLOO, DisARM, BitFlip, straight-through) and produces novel estimators (CRN-VE, CRN-TSMC, MI-SIR, MI-TSMC) that reduce time-adjusted variance by up to 370 imes on queuing, option-pricing and gene-transcription case studies.

Significance. If the results hold, the paper supplies a modular, formally justified design methodology that unifies disparate gradient-estimation techniques under a single reduction to inference + AD, and that systematically generates new low-variance estimators by swapping in stronger inference algorithms. The four main theorems are proved from first principles in a denotational model; full proofs appear in the appendices; a Haskell prototype and a Zenodo artifact accompany the evaluation. The empirical gains (up to 370 imes time-adjusted variance reduction, paired equivalence tests confirming unbiasedness) are concrete and reproducible. These strengths make the work a substantial contribution to both the theory and practice of gradient estimation for probabilistic programs.

major comments (2)
  1. The interchange of derivative and expectation in Eq. (2.6) (and the corresponding hypothesis of Theorem 4) is justified only when the random choices of the inference macro I are independent of the finite perturbation ε. The paper correctly cites existing smoothness analyses and states the condition, but does not give a mechanical check that a user-supplied I satisfies it. For the concrete inference algorithms used in §7 the condition holds by construction; a short clarifying paragraph (or a static check) would make the end-to-end claim fully self-contained for arbitrary I.
  2. §9 Limitations correctly notes that GradInf currently requires the user to annotate each primitive with a factorized coupling. While this is an intentional design choice that enables the modularity of Table 1, the automation claim in the abstract and introduction is therefore slightly overstated. A brief qualification that the system automates the composition of user-supplied couplings (rather than the discovery of the couplings themselves) would align the claims with the delivered system.
minor comments (4)
  1. Figure 2g and Figure 4 report variance and time-adjusted variance; the bootstrap CIs are mentioned in §7.1 but not shown on the plots. Adding them (or stating that they are smaller than marker size) would improve readability.
  2. In Listing 6 the residual flipENUM probability for flipCRN is written with a multi-line fraction that is hard to parse; a single-line expression or an auxiliary let-binding would help.
  3. Table 1 cites “Appendix B.x” for every recovered estimator; a one-sentence pointer in the main text to the corresponding primitive definition would make the recovery claims easier to verify without flipping to the appendix.
  4. The free parameters of the novel estimators (SMC particle counts 2–3, SGD learning rates) are listed in Appendix G; a short note in §7 that these were not tuned against the baselines would forestall any concern about cherry-picking.

Circularity Check

0 steps flagged

No significant circularity; core soundness theorems are proved from first principles via proof-relevant logical relations over QBS, and empirical gains are measured against external baselines.

full rationale

The derivation chain (coupling transformation C, factorized coupling C*, erasure E, partial evaluation S, then composition with any sound inference macro I and AD macro D to form G) is established by Theorems 1–4. These are proved inductively via proof-relevant logical relations R_C and R_S in the denotational model of λ_P / λ_PP over quasi-Borel spaces (Appendices D–F), with no free parameters fitted to data and no reduction of the target gradient identity to an input definition. Recovery of known estimators (Table 1, Appendix B) is by explicit construction of factorized couplings + inference algorithms, not by renaming or definitional identity. Empirical variance reductions (up to 370× time-adjusted) are measured on independently defined models against external baselines (score, SPA phantom, etc.), with bootstrap CIs and paired equivalence tests for unbiasedness; no quantity is fitted then re-predicted. Self-citations (e.g., to ADEV, monad-bayes) supply reusable libraries or prior AD soundness results but are not load-bearing for the uniqueness or correctness of the reduction itself. The interchange condition in (2.6) is an explicit hypothesis of Theorem 4, not a circular assumption. The paper is therefore self-contained against its own claims.

Axiom & Free-Parameter Ledger

2 free parameters · 4 axioms · 3 invented entities

The technical development rests on standard quasi-Borel-space semantics for higher-order probabilistic programs, the measure monad, and ordinary automatic-differentiation correctness results. The only paper-specific inventions are the intermediate language λ_PP, the three source-to-source transformations, and the particular factorized-coupling primitives; all are given explicit denotational semantics and proved sound. Experimental free parameters (particle counts, learning rates) affect only the reported numbers, not the correctness claims.

free parameters (2)
  • SMC particle counts (3 for option pricing, 2 for gene model) = 3 / 2
    Chosen by the authors for the twisted-SMC estimators; affect measured variance but not soundness.
  • SGD learning rates for gene-transcription optimization = tuned per method
    Tuned separately for each method in multiplicative steps of √10; used only for the optimization curves, not for the variance tables.
axioms (4)
  • standard math Quasi-Borel spaces form a Cartesian-closed category that admits a measure monad supporting higher-order probabilistic programs
    Taken from Heunen et al. (2017) and Ścibior et al. (2018); used as the denotational model throughout §§3–6.
  • domain assumption Standard forward- and reverse-mode AD macros correctly differentiate expectations when the random choices of the inference algorithm are independent of the perturbation ε
    Invoked in §6 and Theorem 4; justified by citation to Lee et al. (2023) smoothness analysis and related AD correctness results.
  • domain assumption Any inference macro that preserves the expectation of a Prob Real term yields an unbiased gradient estimator after AD
    Definition of soundness of I{·} (Eq. 6.1) and the composition in Theorem 4.
  • ad hoc to paper The chosen factorized-coupling primitives for each probabilistic primitive satisfy the coupling logical relation after erasure
    Required by the hypotheses of Theorems 2 and 4; verified by direct calculation for each primitive in Listings 3 and 6 and Appendix B.
invented entities (3)
  • λ_PP (factorized probabilistic language with Residual and PProb types) no independent evidence
    purpose: Intermediate representation that enforces non-interference between primal and residual random choices via information-flow typing, enabling partial evaluation.
    Syntactic extension of λ_P based on the dependency core calculus; given full typing rules and denotational interpretation.
  • Coupling, factorization, and partial-evaluation source-to-source transformations C, C*, S no independent evidence
    purpose: Automate the construction of the inference target program from a user-written probabilistic program.
    Defined by induction on syntax; soundness proved by logical relations (Theorems 1–3).
  • Novel gradient estimators GradInf-CRN-VE, GradInf-CRN-TSMC, GradInf-MI-SIR, GradInf-MI-TSMC independent evidence
    purpose: Concrete low-variance estimators obtained by pairing particular couplings with variable elimination or twisted SMC.
    Empirically shown to outperform published baselines; unbiasedness follows from the general theorems.

pith-pipeline@v1.1.0-grok45 · 65878 in / 2906 out tokens · 37786 ms · 2026-07-10T16:56:18.132060+00:00 · methodology

0 comments
read the original abstract

Gradient estimation -- the task of computing the gradient of the expected value of a probabilistic program -- has diverse applications in scientific computing, but is notoriously difficult because of issues such as high-dimensional integration, discrete random choices, and complex stochastic dependencies. This article introduces gradient inference, a new approach to developing sound and efficient gradient estimators for probabilistic programs. Gradient inference rests on a formal reduction from a gradient estimation problem to a closely related probabilistic inference problem, whose solution can be differentiated to obtain a gradient estimator. This inference problem is obtained by applying two powerful statistical operations -- coupling and factorization -- to the input probabilistic program. Our reduction lets us leverage the rich toolkit of probabilistic inference algorithms to design novel gradient estimators that extend and improve upon existing methods. We introduce GradInf, a probabilistic programming system that facilitates the sound and automated implementation of gradient inference. GradInf is centered around programmable source-to-source transformations for coupling and factorizing higher-order probabilistic programs, whose soundness is proven in terms of a denotational semantics. Key to our development is the use of information-flow typing to allow random choices in a probabilistic program to be factored out and partially evaluated, which improves our ability to deploy sophisticated probabilistic inference algorithms. The resulting system offers practitioners a principled framework for designing gradient estimators. We apply GradInf to several challenging case studies, showing that it can express prominent gradient estimators from the literature and enables the construction of new state-of-the-art estimators that outperform the best existing baselines.

Figures

Figures reproduced from arXiv: 2607.07840 by Alexander K. Lew, Feras A. Saad, Gaurav Arya, Mathieu Huot, Moritz Schauer.

Figure 1
Figure 1. Figure 1: Overview of gradient inference workflow using GradInf. The purpose of each component and its [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Applying GradInf to a gradient estimation problem in a discrete-time M/M/c queuing model. [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Overview of program transformations in GradInf. Dotted lines indicate program equiva￾lences, whose properties are stated in each of the respective theorems. The user program is written in 𝜆𝑃 . The probabilistic language 𝜆𝑃𝑃 serves as an intermediate representation for synthesized coupled programs that contain annotations for factorization. Erasing these annotations yields a valid coupled 𝜆𝑃 program. The an… view at source ↗
Figure 4
Figure 4. Figure 4: Variance scaling plots for gradient estimators expressed by GradInf on M/M/c queue model. [PITH_FULL_IMAGE:figures/full_fig_p017_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Applying GradInf to an option pricing model from computational finance. [PITH_FULL_IMAGE:figures/full_fig_p018_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Applying GradInf to infer the hidden parameters of a probabilistic gene transcription model. [PITH_FULL_IMAGE:figures/full_fig_p019_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Gradient estimate samples from the estimators in each case study, with their empirical means reported. [PITH_FULL_IMAGE:figures/full_fig_p020_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Runtime scaling of the score estimator for different gradient estimation systems. [PITH_FULL_IMAGE:figures/full_fig_p020_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: The coupled inference target synthesized by GradInf for a Black-Scholes option pricing model. Applying [PITH_FULL_IMAGE:figures/full_fig_p031_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Discrete-latent benchmark models and training loss curves for VAE gradient inference experiments. [PITH_FULL_IMAGE:figures/full_fig_p036_10.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

88 extracted references · 88 canonical work pages · 9 internal anchors

  1. [1]

    Martín Abadi, Anindya Banerjee, Nevin Heintze, and Jon G. Riecke. 1999. A Core Calculus of Dependency. InProceedings of the 26th ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages. Association for Computing Machinery, New York, NY, 147–160. https://doi.org/10.1145/292540.292555

    work page doi:10.1145/292540.292555 1999

  2. [2]

    Alejandro Aguirre, Gilles Barthe, Lars Birkedal, Aleš Bizjak, Marco Gaboardi, and Deepak Garg. 2018. Relational Reasoning for Markov Chains in a Probabilistic Guarded Lambda Calculus. InProceedings of the 27th European Symposium on Programming. Springer, Cham, 214–241. https://doi.org/10.1007/978-3-319-89884-1_8

    work page doi:10.1007/978-3-319-89884-1_8 2018

  3. [3]

    Amal Ahmed. 2006. Step-Indexed Syntactic Logical Relations for Recursive and Quantified Types. InProceedings of the 15th European Symposium on Programming, Peter Sestoft (Ed.), Vol. 3924. Springer Berlin Heidelberg, Berlin, Heidelberg, 69–83. https://doi.org/10.1007/11693024_6

    work page doi:10.1007/11693024_6 2006

  4. [4]

    Anderson

    David F. Anderson. 2012. An Efficient Finite Difference Method for Parameter Sensitivities of Continuous Time Markov Chains.SIAM J. Numer. Anal.50, 5 (Jan. 2012), 2237–2258. https://doi.org/10.1137/110849079

    work page doi:10.1137/110849079 2012

  5. [5]

    GradInf: Gradient Estimation as Probabilistic Inference

    Gaurav Arya, Mathieu Huot, Moritz Schauer, Alexander Lew, and Feras Saad. 2026.Artifact for "GradInf: Gradient Estimation as Probabilistic Inference". https://doi.org/10.5281/zenodo.19673572

    work page doi:10.5281/zenodo.19673572 2026

  6. [6]

    Gaurav Arya, Moritz Schauer, Frank Schäfer, and Christopher Rackauckas. 2022. Automatic Differentiation of Programs with Discrete Randomness. InProceedings of the 36th International Conference on Neural Information Processing Systems (Advances in Neural Information Processing Systems, Vol. 35). Curran Associates, Inc., Red Hook, NY, Article 758, 13 pages....

    work page doi:10.5555/3600270.3601028 2022

  7. [7]

    Sai Praveen Bangaru, Michael Gharbi, Fujun Luan, Tzu-Mao Li, Kalyan Sunkavalli, Milos Hasan, Sai Bi, Zexiang Xu, Gilbert Bernstein, and Fredo Durand. 2022. Differentiable Rendering of Neural SDFs through Reparameterization. In SIGGRAPH Asia 2022 Conference Papers. Association for Computing Machinery, New York, NY, USA, Article 5, 9 pages. https://doi.org/...

    work page doi:10.1145/3550469.3555397 2022

  8. [8]

    Gilles Barthe, Raphaëlle Crubillé, Ugo Dal Lago, and Francesco Gavazzo. 2020. On the Versatility of Open Logical Relations: Continuity, Automatic Differentiation, and a Containment Theorem. InProceedings of the 29th European Symposium on Programming. Springer-Verlag, Berlin, 56–83. https://doi.org/10.1007/978-3-030-44914-8_3

    work page doi:10.1007/978-3-030-44914-8_3 2020

  9. [9]

    Gilles Barthe, Thomas Espitau, Benjamin Grégoire, Justin Hsu, Léo Stefanesco, and Pierre-Yves Strub. 2015. Relational Reasoning via Probabilistic Coupling. InProceedings of the 20th International Conference on Logic for Programming, Artificial Intelligence, and Reasoning. Springer-Verlag, Berlin, Heidelberg, 387–401. https://doi.org/10.1007/978-3-662- 48899-7_27

    work page doi:10.1007/978-3-662- 2015

  10. [10]

    Gilles Barthe, Benjamin Grégoire, Justin Hsu, and Pierre-Yves Strub. 2017. Coupling Proofs Are Probabilistic Product Programs. InProceedings of the 44th ACM SIGPLAN Symposium on Principles of Programming Languages. Association for Computing Machinery, New York, NY, USA, 161–174. https://doi.org/10.1145/3009837.3009896

    work page doi:10.1145/3009837.3009896 2017

  11. [11]

    Pearlmutter, Alexey Andreyevich Radul, and Jeffrey Mark Siskind

    Atılım Günes Baydin, Barak A. Pearlmutter, Alexey Andreyevich Radul, and Jeffrey Mark Siskind. 2017. Automatic Differentiation in Machine Learning: A Survey.Journal of Machine Learning Research18, 1 (Jan. 2017), 5595—-5637. https://doi.org/10.5555/3122009.3242010

    work page doi:10.5555/3122009.3242010 2017

  12. [12]

    Becker, Alexander K

    McCoy R. Becker, Alexander K. Lew, Xiaoyan Wang, Matin Ghavami, Mathieu Huot, Martin C. Rinard, and Vikash K. Mansinghka. 2024. Probabilistic Programming with Programmable Variational Inference.Proceedings of the ACM on Programming Languages8, PLDI (June 2024), 2123–2147. https://doi.org/10.1145/3656463

    work page doi:10.1145/3656463 2024

  13. [13]

    Yoshua Bengio, Nicholas Léonard, and Aaron Courville. 2013. Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation. arXiv:1308.3432

    work page internal anchor Pith review Pith/arXiv arXiv 2013

  14. [14]

    Chen, Martin Jankowiak, Fritz Obermeyer, Neeraj Pradhan, Theofanis Karaletsos, Rohit Singh, Paul Szerlip, Paul Horsfall, and Noah D

    Eli Bingham, Jonathan P. Chen, Martin Jankowiak, Fritz Obermeyer, Neeraj Pradhan, Theofanis Karaletsos, Rohit Singh, Paul Szerlip, Paul Horsfall, and Noah D. Goodman. 2019. Pyro: Deep Universal Probabilistic Programming. Journal of Machine Learning Research20, 28 (2019), 1–6. https://doi.org/10.5555/3322706.3322734

    work page doi:10.5555/3322706.3322734 2019

  15. [15]

    Jörg Bornschein and Yoshua Bengio. 2015. Reweighted Wake-Sleep. InInternational Conference on Learning Represen- tations. 12 pages. arXiv:1406.2751 Proc. ACM Program. Lang., Vol. 10, No. PLDI, Article 243. Publication date: June 2026. 243:24 Gaurav Arya, Mathieu Huot, Moritz Schauer, Alexander K. Lew, and Feras A. Saad

    work page internal anchor Pith review Pith/arXiv arXiv 2015

  16. [16]

    Phelim P. Boyle. 1988. A Lattice Framework for Option Pricing with Two State Variables.Journal of Financial and Quantitative Analysis23, 1 (March 1988), 1–12. https://doi.org/10.2307/2331019

    work page doi:10.2307/2331019 1988

  17. [17]

    Chandran, W.B

    D. Chandran, W.B. Copeland, S.C. Sleight, and H.M. Sauro. 2008. Mathematical Modeling and Synthetic Biology.Drug Discovery Today: Disease Models5, 4 (2008), 299–309. https://doi.org/10.1016/j.ddmod.2009.07.002

    work page doi:10.1016/j.ddmod.2009.07.002 2008

  18. [18]

    2020.An Introduction to Sequential Monte Carlo

    Nicolas Chopin and Omiros Papaspiliopoulos. 2020.An Introduction to Sequential Monte Carlo. Springer, Cham. https://doi.org/10.1007/978-3-030-47845-2

    work page doi:10.1007/978-3-030-47845-2 2020

  19. [19]

    Aditya Prakash, and Ramesh Raskar

    Ayush Chopra, Alexander Rodríguez, Jayakumar Subramanian, Arnau Quera-Bofarull, Balaji Krishnamurthy, B. Aditya Prakash, and Ramesh Raskar. 2023. Differentiable Agent-based Epidemiology. InProceedings of the 2023 International Conference on Autonomous Agents and Multiagent Systems. International Foundation for Autonomous Agents and Multiagent Systems, Ric...

    work page doi:10.5555/3545946.3598851 2023

  20. [20]

    Joel S. Cohen. 2002.Computer Algebra and Symbolic Computation: Elementary Algorithms. A K Peters, Natick, MA

    work page 2002

  21. [21]

    Cusumano-Towner, Feras A

    Marco F. Cusumano-Towner, Feras A. Saad, Alexander K. Lew, and Vikash K. Mansinghka. 2019. Gen: A General- Purpose Probabilistic Programming System with Programmable Inference. InProceedings of the 40th ACM SIGPLAN Conference on Programming Design and Implementation. Association for Computing Machinery, New York, NY, 221–236. https://doi.org/10.1145/33142...

    work page doi:10.1145/3314221.3314642 2019

  22. [22]

    Liyi Dai. 2000. Perturbation Analysis via Coupling.IEEE Trans. Automat. Control45, 4 (April 2000), 614–628. https://doi.org/10.1109/9.847099

    work page doi:10.1109/9.847099 2000

  23. [23]

    TensorFlow Distributions

    Joshua V. Dillon, Ian Langmore, Dustin Tran, Eugene Brevdo, Vasudevan Srinivas, Dave Moore, Brian Patton, Alex Alemi, Matthew D. Hoffman, and Rif A. Saurous. 2017. TensorFlow Distributions. arXiv:1711.10604

    work page internal anchor Pith review Pith/arXiv arXiv 2017

  24. [24]

    Zhe Dong, Andriy Mnih, and George Tucker. 2020. DisARM: An Antithetic Gradient Estimator for Binary Latent Variables. InProceedings of the 34th International Conference on Neural Information Processing Systems (Advances in Neural Information Processing Systems, Vol. 33). Curran Associates, Inc., Red Hook, NY, 18637–18647

    work page 2020

  25. [25]

    1997.Conditional Monte Carlo: Gradient Estimation Optimization and Applications

    Michael Fu and Jian-Qiang Hu. 1997.Conditional Monte Carlo: Gradient Estimation Optimization and Applications. Springer International Series in Engineering and Computer Science, Vol. 392. Springer, New York. https://doi.org/10. 1007/978-1-4615-6293-1

    work page 1997

  26. [26]

    Gillespie

    Daniel T. Gillespie. 1977. Exact Stochastic Simulation of Coupled Chemical Reactions.The Journal of Physical Chemistry 81, 25 (Dec. 1977), 2340–2361. https://doi.org/10.1021/j100540a008

    work page doi:10.1021/j100540a008 1977

  27. [27]

    Michèle Giry. 1982. A Categorical Approach to Probability Theory. InCategorical Aspects of Topology and Analysis, B. Banaschewski (Ed.). Springer, Berlin, Heidelberg, 68–85. https://doi.org/10.1007/BFb0092872

    work page doi:10.1007/bfb0092872 1982

  28. [28]

    2003.Monte Carlo Methods in Financial Engineering

    Paul Glasserman. 2003.Monte Carlo Methods in Financial Engineering. Number 53 in Stochastic Modelling and Applied Probability. Springer, New York, NY. https://doi.org/10.1007/978-0-387-21617-1

    work page doi:10.1007/978-0-387-21617-1 2003

  29. [29]

    1991.Gradient Estimation via Perturbation Analysis

    Paul Glasserman and Yu-Chi Ho. 1991.Gradient Estimation via Perturbation Analysis. Number 116 in Kluwer International Series in Engineering and Computer Science Discrete Event Dynamic Systems. Kluwer Academic Publ, Boston

    work page 1991

  30. [30]

    Peter W. Glynn. 1990. Likelihood Ratio Gradient Estimation for Stochastic Systems.Commun. ACM33, 10 (1990), 75–84. https://doi.org/10.1145/84537.84552

    work page doi:10.1145/84537.84552 1990

  31. [31]

    Wei-Bo Gong and Yu-Chi Ho. 1987. Smoothed (Conditional) Perturbation Analysis of Discrete-Event Dynamic Systems. IEEE Trans. Automat. Control32, 10 (Oct. 1987), 858–866. https://doi.org/10.1109/TAC.1987.1104464

    work page doi:10.1109/tac.1987.1104464 1987

  32. [32]

    Gorinova, Andrew D

    Maria I. Gorinova, Andrew D. Gordon, and Charles Sutton. 2019. Probabilistic Programming with Densities in SlicStan: Efficient, Flexible, and Deterministic.Proceedings of the ACM on Programming Languages3, POPL (Jan. 2019), 1–30. https://doi.org/10.1145/3290348

    work page doi:10.1145/3290348 2019

  33. [33]

    Gorinova, Andrew D

    Maria I. Gorinova, Andrew D. Gordon, Charles Sutton, and Matthijs Vákár. 2022. Conditional Independence by Typing. ACM Transactions on Programming Languages and Systems44, 1 (2022), 1–54. https://doi.org/10.1145/3490421

    work page doi:10.1145/3490421 2022

  34. [34]

    Haselwarter, Joseph Tassarotti, and Lars Birkedal

    Simon Oddershede Gregersen, Alejandro Aguirre, Philipp G. Haselwarter, Joseph Tassarotti, and Lars Birkedal. 2024. Asynchronous Probabilistic Couplings in Higher-Order Separation Logic.Proceedings of the ACM on Programming Languages8, POPL (Jan. 2024), 753–784. https://doi.org/10.1145/3632868

    work page doi:10.1145/3632868 2024

  35. [35]

    Haselwarter, Kwing Hei Li, Alejandro Aguirre, Simon Oddershede Gregersen, Joseph Tassarotti, and Lars Birkedal

    Philipp G. Haselwarter, Kwing Hei Li, Alejandro Aguirre, Simon Oddershede Gregersen, Joseph Tassarotti, and Lars Birkedal. 2025. Approximate Relational Reasoning for Higher-Order Probabilistic Programs.Proceedings of the ACM on Programming Languages9, POPL (Jan. 2025), 1196–1226. https://doi.org/10.1145/3704877

    work page doi:10.1145/3704877 2025

  36. [36]

    Heidergott and F

    B. Heidergott and F. J. Vázquez-Abad. 2008. Measure-Valued Differentiation for Markov Chains.Journal of Optimization Theory and Applications136, 2 (Feb. 2008), 187–209. https://doi.org/10.1007/s10957-007-9297-7

    work page doi:10.1007/s10957-007-9297-7 2008

  37. [37]

    Chris Heunen, Ohad Kammar, Sam Staton, and Hongseok Yang. 2017. A Convenient Category for Higher-Order Probability Theory. InProceedings of the 32nd Annual Symposium on Logic in Computer Science. IEEE Press, Piscataway, NJ, 12 pages. https://doi.org/10.1109/LICS.2017.8005137

    work page doi:10.1109/lics.2017.8005137 2017

  38. [38]

    Steven Holtzen, Guy Van den Broeck, and Todd Millstein. 2020. Scaling Exact Inference for Discrete Probabilistic Programs.Proceedings of the ACM on Programming Languages4, OOPSLA, Article 140 (Nov. 2020), 31 pages. https: //doi.org/10.1145/3428208 Proc. ACM Program. Lang., Vol. 10, No. PLDI, Article 243. Publication date: June 2026. GradInf: Gradient Esti...

    work page doi:10.1145/3428208 2020

  39. [39]

    Mathieu Huot, Sam Staton, and Matthijs Vákár. 2020. Correctness of Automatic Differentiation via Diffeologies and Categorical Gluing. InFoundations of Software Science and Computation Structures, Jean Goubault-Larrecq and Barbara König (Eds.). Springer International Publishing, Cham, 319–338. https://doi.org/10.1007/978-3-030-45231-5_17

    work page doi:10.1007/978-3-030-45231-5_17 2020

  40. [40]

    Michael Kagan and Lukas Heinrich. 2023. Branches of a Tree: Taking Derivatives of Programs with Discrete and Branching Randomness in High Energy Physics. arXiv:2308.16680

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  41. [41]

    Luke Ong, and Dominik Wagner

    Basim Khajwal, C.-H. Luke Ong, and Dominik Wagner. 2023. Fast and Correct Gradient-Based Optimisation for Probabilistic Programming via Smoothing. InProceedings of the 32nd European Symposium on Programming. Springer- Verlag, Berlin, 479–506. https://doi.org/10.1007/978-3-031-30044-8_18

    work page doi:10.1007/978-3-031-30044-8_18 2023

  42. [42]

    backpropagation

    Lê Anh Khoa. 2025. mstksg/backprop: Heterogeneous Automatic Differentiation (“backpropagation”) in Haskell. https://github.com/mstksg/backprop

    work page 2025

  43. [43]

    Auto-Encoding Variational Bayes

    Diederik P. Kingma and Max Welling. 2013. Auto-Encoding Variational Bayes. arXiv:1312.6114

    work page internal anchor Pith review Pith/arXiv arXiv 2013

  44. [44]

    Edward Kmett. 2025. ekmett/ad: Automatic Differentiation. https://github.com/ekmett/ad

    work page 2025

  45. [45]

    2009.Probabilistic Graphical Models: Principles and Techniques

    Daphne Koller and Nir Friedman. 2009.Probabilistic Graphical Models: Principles and Techniques. MIT Press, Cambridge, MA

    work page 2009

  46. [46]

    Eisenberg, and Andrew Fitzgibbon

    Faustyna Krawiec, Simon Peyton Jones, Neel Krishnaswami, Tom Ellis, Richard A. Eisenberg, and Andrew Fitzgibbon

    work page

  47. [47]

    2022), 30 pages

    Provably Correct, Asymptotically Efficient, Higher-Order Reverse-Mode Automatic Differentiation.Proceedings of the ACM on Programming Languages6, POPL, Article 48 (Jan. 2022), 30 pages. https://doi.org/10.1145/3498710

    work page doi:10.1145/3498710 2022

  48. [48]

    Kunes, Mingzhang Yin, Max Land, Doron Haviv, Dana Pe’er, and Simon Tavaré

    Russell Z. Kunes, Mingzhang Yin, Max Land, Doron Haviv, Dana Pe’er, and Simon Tavaré. 2023. Gradient Estimation for Binary Latent Variables via Gradient Variance Clipping.Proceedings of the AAAI Conference on Artificial Intelligence 37, 7 (June 2023), 8405–8412. https://doi.org/10.1609/aaai.v37i7.26013

    work page doi:10.1609/aaai.v37i7.26013 2023

  49. [49]

    Scheel, and Peder M

    Daniël Lakens, Anne M. Scheel, and Peder M. Isager. 2018. Equivalence Testing for Psychological Research: A Tutorial.Advances in Methods and Practices in Psychological Science1, 2 (2018), 259–269. https://doi.org/10.1177/ 2515245918770963

    work page 2018

  50. [50]

    Larsen and Arne Skou

    Kim G. Larsen and Arne Skou. 1991. Bisimulation Through Probabilistic Testing.Information and Computation94, 1 (1991), 1–28. https://doi.org/10.1016/0890-5401(91)90030-6

    work page doi:10.1016/0890-5401(91)90030-6 1991

  51. [51]

    Tuan Anh Le, Maximilian Igl, Tom Rainforth, Tom Jin, and Frank Wood. 2018. Auto-Encoding Sequential Monte Carlo. InInternational Conference on Learning Representations. 10 pages. arXiv:1705.10306

    work page internal anchor Pith review Pith/arXiv arXiv 2018

  52. [52]

    Kosiorek, N

    Tuan Anh Le, Adam R. Kosiorek, N. Siddharth, Yee Whye Teh, and Frank Wood. 2020. Revisiting Reweighted Wake- Sleep for Models with Stochastic Control Flow. InProceedings of the 35th Uncertainty in Artificial Intelligence Conference. PMLR, Norfolk, MA, 1039–1049

    work page 2020

  53. [53]

    Wonyeol Lee, Xavier Rival, and Hongseok Yang. 2023. Smoothness Analysis for Probabilistic Programs with Application to Optimised Variational Inference.Proceedings of the ACM on Programming Languages7, POPL, Article 12 (Jan. 2023), 32 pages. https://doi.org/10.1145/3571205

    work page doi:10.1145/3571205 2023

  54. [54]

    Wonyeol Lee, Hangyeol Yu, Xavier Rival, and Hongseok Yang. 2020. On Correctness of Automatic Differentiation for Non-Differentiable Functions. InProceedings of the 34th International Conference on Neural Information Processing Systems (Advances in Neural Information Processing Systems, Vol. 33). Curran Associates Inc., Red Hook, NY, USA, Article 564, 12 pages

    work page 2020

  55. [55]

    Lew, Marco F

    Alexander K. Lew, Marco F. Cusumano-Towner, Benjamin Sherman, Michael Carbin, and Vikash K. Mansinghka. 2020. Trace Types and Denotational Semantics for Sound Programmable Inference in Probabilistic Languages.Proceedings of the ACM on Programming Languages4, POPL (Jan. 2020), 1–32. https://doi.org/10.1145/3371087

    work page doi:10.1145/3371087 2020

  56. [56]

    Lew, Mathieu Huot, Sam Staton, and Vikash K

    Alexander K. Lew, Mathieu Huot, Sam Staton, and Vikash K. Mansinghka. 2023. ADEV: Sound Automatic Differentiation of Expected Values of Probabilistic Programs.Proceedings of the ACM on Programming Languages7, POPL (Jan. 2023), 121–153. https://doi.org/10.1145/3571198

    work page doi:10.1145/3571198 2023

  57. [57]

    Jianlin Li, Eric Wang, and Yizhou Zhang. 2024. Compiling Probabilistic Programs for Variable Elimination with Information Flow.Proceedings of the ACM on Programming Languages8, PLDI (June 2024), 1755–1780. https: //doi.org/10.1145/3656448

    work page doi:10.1145/3656448 2024

  58. [58]

    Tzu-Mao Li, Miika Aittala, Frédo Durand, and Jaakko Lehtinen. 2018. Differentiable Monte Carlo Ray Tracing Through Edge Sampling.ACM Trans. Graph.37, 6, Article 222 (Dec. 2018), 11 pages. https://doi.org/10.1145/3272127.3275109

    work page doi:10.1145/3272127.3275109 2018

  59. [59]

    2002.Lectures on the Coupling Method

    Torgny Lindvall. 2002.Lectures on the Coupling Method. Dover Publications, Mineola, NY

    work page 2002

  60. [60]

    Jun S. Liu. 2004.Monte Carlo Strategies in Scientific Computing(1 ed.). Springer, New York, NY, USA. https: //doi.org/10.1007/978-0-387-76371-2

    work page doi:10.1007/978-0-387-76371-2 2004

  61. [61]

    Runjing Liu, Jeffrey Regier, Nilesh Tripuraneni, Michael Jordan, and Jon Mcauliffe. 2019. Rao-Blackwellized Stochastic Gradients for Discrete Distributions. InProceedings of the 36th International Conference on Machine Learning. PMLR, Norfolk, MA, 4023–4031

    work page 2019

  62. [62]

    Maddison, Dieterich Lawson, George Tucker, Nicolas Heess, Mohammad Norouzi, Andriy Mnih, Arnaud Doucet, and Yee Whye Teh

    Chris J. Maddison, Dieterich Lawson, George Tucker, Nicolas Heess, Mohammad Norouzi, Andriy Mnih, Arnaud Doucet, and Yee Whye Teh. 2017. Filtering Variational Objectives. InProceedings of the 31st International Conference on Neural Information Processing Systems (Advances in Neural Information Processing Systems, Vol. 30). Curran Associates, Proc. ACM Pro...

    work page 2017

  63. [63]

    Mansinghka, Ulrich Schaechtle, Shivam Handa, Alexey Radul, Yutian Chen, and Martin Rinard

    Vikash K. Mansinghka, Ulrich Schaechtle, Shivam Handa, Alexey Radul, Yutian Chen, and Martin Rinard. 2018. Probabilistic Programming with Programmable Inference. InProceedings of the 39th ACM SIGPLAN Conference on Programming Design and Implementation. Association for Computing Machinery, New York, NY, 603–616. https: //doi.org/10.1145/3192366.3192409

    work page doi:10.1145/3192366.3192409 2018

  64. [64]

    Jesse Michel, Kevin Mu, Xuanda Yang, Sai Praveen Bangaru, Elias Rojas Collins, Gilbert Bernstein, Jonathan Ragan-Kelley, Michael Carbin, and Tzu-Mao Li. 2024. Distributions for Compositionally Differentiating Para- metric Discontinuities.Proceedings of the ACM on Programming Languages8, OOPSLA1 (April 2024), 893–922. https://doi.org/10.1145/3649843

    work page doi:10.1145/3649843 2024

  65. [65]

    Andriy Mnih and Danilo J. Rezende. 2016. Variational Inference for Monte Carlo Objectives. arXiv:1602.06725

    work page internal anchor Pith review Pith/arXiv arXiv 2016

  66. [66]

    Shakir Mohamed, Mihaela Rosca, Michael Figurnov, and Andriy Mnih. 2020. Monte Carlo Gradient Estimation in Machine Learning.Journal of Machine Learning Research21, 132 (2020), 1–62. https://doi.org/10.5555/3455716.3455848

    work page doi:10.5555/3455716.3455848 2020

  67. [67]

    Christian Naesseth, Scott Linderman, Rajesh Ranganath, and David Blei. 2018. Variational Sequential Monte Carlo. In Proceedings of the 21st International Conference on Artificial Intelligence and Statistics (Proceedings of Machine Learning Research, Vol. 84), Amos Storkey and Fernando Perez-Cruz (Eds.). PMLR, 968–977

    work page 2018

  68. [68]

    Praveen Narayanan and Chung-chieh Shan. 2020. Symbolic Disintegration with a Variety of Base Measures.ACM Transactions on Programming Languages and Systems42, 2, Article 9 (2020), 60 pages. https://doi.org/10.1145/3374208

    work page doi:10.1145/3374208 2020

  69. [69]

    2011.The Art of Differentiating Computer Programs

    Uwe Naumann. 2011.The Art of Differentiating Computer Programs. Society for Industrial and Applied Mathematics, Philadelphia. https://doi.org/10.1137/1.9781611972078

    work page doi:10.1137/1.9781611972078 2011

  70. [70]

    Art B. Owen. 2013.Monte Carlo Theory, Methods and Examples. https://artowen.su.domains/mc/

    work page 2013

  71. [71]

    Viktor Pfanschilling, Hikaru Shindo, Devendra Singh Dhami, and Kristian Kersting. 2022. Sum-Product Loop Pro- gramming: From Probabilistic Circuits to Loop Programming. InProceedings of the 19th International Conference on Principles of Knowledge Representation and Reasoning. IJCAI Organization, 453–462. https://doi.org/10.24963/kr.2022/47

    work page doi:10.24963/kr.2022/47 2022

  72. [72]

    Norman Ramsey and Avi Pfeffer. 2002. Stochastic Lambda Calculus and Monads of Probability Distributions. In Proceedings of the 29th ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages. Association for Computing Machinery, New York, NY, USA, 154–165. https://doi.org/10.1145/503272.503288

    work page doi:10.1145/503272.503288 2002

  73. [73]

    Saad, Martin C

    Feras A. Saad, Martin C. Rinard, and Vikash K. Mansinghka. 2021. SPPL: Probabilistic Programming with Fast Exact Symbolic Inference. InProceedings of the 42nd ACM SIGPLAN International Conference on Programming Language Design and Implementation. Association for Computing Machinery, New York, NY, USA, 804–819. https://doi.org/10. 1145/3453483.3454078

    work page arXiv 2021

  74. [74]

    Tetsuya Sato, Alejandro Aguirre, Gilles Barthe, Marco Gaboardi, Deepak Garg, and Justin Hsu. 2019. Formal Verification of Higher-Order Probabilistic Programs: Reasoning about Approximation, Convergence, Bayesian Inference, and Optimization.Proceedings of the ACM on Programming Languages3, POPL (Jan. 2019), 1–30. https://doi.org/10.1145/ 3290351

    work page 2019

  75. [75]

    Tetsuya Sato, Gilles Barthe, Marco Gaboardi, Justin Hsu, and Shin-ya Katsumata. 2019. Approximate Span Liftings: Compositional Semantics for Relaxations of Differential Privacy. InProceedings of the 34th Annual Symposium on Logic in Computer Science (LICS ’19). IEEE Press, 1–14. https://doi.org/10.1109/LICS.2019.8785668

    work page doi:10.1109/lics.2019.8785668 2019

  76. [76]

    John Schulman, Nicolas Heess, Theophane Weber, and Pieter Abbeel. 2015. Gradient Estimation Using Stochastic Computation Graphs. InProceedings of the 29th International Conference on Neural Information Processing Systems (Advances in Neural Information Processing Systems, Vol. 28). Curran Associates, Inc., Red Hook, NY, 3528–3536

    work page 2015

  77. [77]

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017. Proximal Policy Optimization Algorithms. arXiv:arXiv:1707.06347

    work page internal anchor Pith review Pith/arXiv arXiv 2017

  78. [78]

    Adam Ścibior, Ohad Kammar, and Zoubin Ghahramani. 2018. Functional Programming for Modular Bayesian Inference. Proceedings of the ACM on Programming Languages2, ICFP, Article 83 (July 2018), 29 pages. https://doi.org/10.1145/ 3236778

    work page 2018

  79. [79]

    Moss, Chris Heunen, and Zoubin Ghahramani

    Adam Ścibior, Ohad Kammar, Matthijs Vákár, Sam Staton, Hongseok Yang, Yufei Cai, Klaus Ostermann, Sean K. Moss, Chris Heunen, and Zoubin Ghahramani. 2018. Denotational Validation of Higher-Order Bayesian Inference.Proceedings of the ACM on Programming Languages2, POPL, Article 60 (Jan. 2018), 29 pages. https://doi.org/10.1145/3158148

    work page doi:10.1145/3158148 2018

  80. [80]

    Richard Statman. 1985. Logical Relations and the Typed 𝜆-Calculus.Information and Control65, 2 (May 1985), 85–97. https://doi.org/10.1016/S0019-9958(85)80001-2

    work page doi:10.1016/s0019-9958(85)80001-2 1985

Showing first 80 references.