Pith. sign in

REVIEW 3 major objections 5 minor 78 references

A Markov chain sampler that follows light-like rays through a likelihood-dependent refractive index can draw fair posterior samples from billion-parameter neural networks on a single consumer GPU.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

T0 review · deepseek-v4-flash

2026-08-04 07:25 UTC pith:56G4R4BV

load-bearing objection A genuinely new sampler with an attractive noise-resilience story and an honest, well-documented empirical section; the advertised ability to cross zero-likelihood holes is not proven, but the core method deserves a serious referee. the 3 major comments →

arxiv 2510.25824 v2 pith:56G4R4BV submitted 2025-10-29 astro-ph.IM stat.ML

The Ray Tracing Sampler: Bayesian Sampling of Neural Networks for Everyone

classification astro-ph.IM stat.ML
keywords ray tracing samplerMarkov chain Monte Carlostochastic gradient noiseBayesian neural networksHamiltonian Monte CarloSnell's lawhigh-dimensional sampling
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper introduces an MCMC method in which a walker follows ray paths through parameter space whose refractive index is a power of the likelihood. By choosing the refractive index as the likelihood raised to the power 1/(D-1) in D dimensions, the geometric density of ray bundles becomes proportional to the likelihood, giving fair sampling; a Metropolis-like acceptance test restores exactness when the integrator errs. The central claim is that this makes Bayesian sampling of neural-network posteriors practical at billion-parameter scale on ordinary hardware, because the method tolerates far more stochastic-gradient noise than Hamiltonian Monte Carlo and can cross arbitrary likelihood barriers. A sympathetic reader would care because it offers a route to uncertainty quantification for models that are currently only trained, not sampled.

Core claim

The paper's central discovery is that ray tracing with refractive index n(x)=L(x)^(1/(D-1)) produces a Markov chain whose stationary distribution is proportional to the likelihood L(x). The proof rests on conservation of basic radiance, R = n^(1-D) L, and étendue along ray bundles: Snell's law compresses the ray-bundle phase space in exactly the right way to cancel the likelihood difference between starting and ending points. For imperfect integrators, exactness is restored by the acceptance criterion P(x→y) = min(1, L(y)L(x)/(L(x)L(y))), where L is the ray-bundle radiance. The paper further claims that this sampler is far more resistant to heating from stochastic gradients than Hamiltonian

What carries the argument

The load-bearing identity is the conservation of basic radiance R = n^(1-D) L and its companion étendue invariant G = n^(D-1) dA cosθ dΩ along ray paths. Choosing n(x)=L(x)^(1/(D-1)) makes the radiance L proportional to the target likelihood L(x), so the ray density is the target density. The paper uses Snell's law in differential form, dθ/ds = -sinθ |∇ ln n(x)|, to update propagation direction, and the acceptance test based on the basic-radiance ratio to correct for integrator error.

Load-bearing premise

The claim that the sampler can cross any likelihood barrier rests on the assumption that a single step of length ε+δ from a point arbitrarily close to a boundary exits the reachable region; because ray directions bend continuously, the ray can be deflected away from the boundary before that distance is traveled, potentially leaving the sampler trapped in one mode.

What would settle it

Run the ray-tracing sampler on a two-mode target distribution separated by a broad, deep valley, using a small step size and no momentum refresh. If the chain essentially never leaves one mode in the long-time limit, while the local acceptance test continues to pass, the barrier-crossing claim is falsified.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Bayesian posterior sampling becomes feasible for neural networks with over a billion parameters on a single consumer GPU, not just on large compute clusters.
  • Stochastic mini-batch gradients cause orders of magnitude less chain heating than in Hamiltonian Monte Carlo, so small batch sizes can be used without runaway divergence.
  • The sampler's acceptance test makes the resulting distribution fair even when the numerical integrator is imperfect.
  • The method can cross arbitrary likelihood barriers and even holes in parameter space, which Hamiltonian-style methods generally cannot.
  • Prior algorithms, including Hamiltonian Monte Carlo, microcanonical HMC, Metropolis, Gibbs, and Monte Carlo integration, reappear as special cases of the generalized ray tracing framework.
  • The generalized weighting scheme allows sampling according to an arbitrary weighting function and supports likelihood-dependent tempering along the path.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the stochastic-gradient resilience holds at scale, practitioners can shrink mini-batch sizes by large factors without paying the compute penalty HMC incurs, so sampling and training can share a single compute budget.
  • The paper's own demonstrations on large models are approximate: convergence is verified in function space, not weight space, and the GPT-2 run shows quantization spikes and an effective parameter count far below the nominal count. A fair reader should expect the 'everyone' promise to depend on function-space convergence tests.
  • The information-flow measure the paper introduces for comparing network architectures could be reused as an objective for automated architecture search, though the paper presents it only as a diagnostic.
  • Because standard HMC is a special case of generalized ray tracing, the framework suggests an alternative route to stochastic HMC: instead of adding a cooling term, keep the ray speed fixed and reweight samples post hoc, which could sidestep the difficulty of tuning friction parameters.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces a new MCMC family based on ray propagation in a medium with refractive index chosen as a function of the desired likelihood. The main theoretical claim is that the conserved quantity of geometric optics — basic radiance — can be used to construct a Markov chain with stationary distribution proportional to L(x). Concretely, the authors set n(x)=L(x)^{1/(D-1)} and propose a deterministic, reversible integrator with a Metropolis-style acceptance step (Eq. 17) intended to correct discretization error. The paper further claims ergodicity and the ability to cross arbitrary likelihood barriers, including parameter-space holes, and demonstrates the sampler on a 10,000-dimensional Gaussian, a 1,433-parameter MLP, a 22-million-parameter ResNet, and a 1.5-billion-parameter GPT-2 model on a single consumer GPU. It also presents a generalized ray-tracing framework in which HMC, microcanonical HMC, Gibbs, Metropolis, and Monte Carlo integration appear as special cases.

Significance. If the central stationarity claim is fully established, this is a novel and potentially important contribution: the geometric-optics derivation is elegant, the idea of using radiance conservation rather than energy conservation as a basis for exact MCMC is original, and the stochastic-gradient robustness results (§3.1.2) are striking. The public implementation in C, JAX, and PyTorch is a concrete strength, as is the honest Appendix D showing that ray tracing does not dramatically outperform HMC on standard distributions with perfect gradients. The GPT-2 experiment, although approximate, is a useful scalability demonstration. However, the paper currently asserts rather than proves a key ingredient of the discrete algorithm — the Jacobian of its integrator — and several advertised properties, especially hole-crossing ergodicity, are not supported by the arguments given.

major comments (3)
  1. [§2.4, Eqs. (16)–(17); §2.8, Algorithm 1] The paper claims that the acceptance probability P(x→y)=min(1, L(y)L(x)/(L(x)L(y))) guarantees exact sampling even with imperfect integrators. The radiance-conservation derivation applies to continuous ray bundles crossing interfaces; Algorithm 1, however, uses a discrete drift-kick-drift map in which the direction is updated at a midpoint while position advances along straight segments. For a reversible non-volume-preserving map, a Metropolis-Hastings correction must include the Jacobian of the proposal transformation. The manuscript does not compute det ∂(y,v′)/∂(x,v) for the actual integrator, nor does it prove that this determinant equals the sine-ratio product in Eq. (16). Without this step, the central claim of exact sampling for imperfect integrators is unsupported. I recommend adding a direct derivation or a numerical verification of the Jacobian for one integration step, followe
  2. [§2.5; Appendix A; abstract] The ergodicity and hole-crossing claims are not established. The proof in §2.5 assumes that from a point arbitrarily close to the boundary a step of length ε+δ exits the reachable set. This requires the ray equation to remain nondegenerate. When the 'hole' is a region with L=0, the refractive index n=L^{1/(D-1)} vanishes and the ray equation degenerates; there is no continuous ray path into the zero-index region, so the argument fails. The blackbody-radiator treatment in Appendix A is not part of Algorithm 1, and no stationarity proof is given for it. Thus the abstract's claim of crossing 'including holes in parameter space' is an overclaim. The core sampler may be valid for smoothly varying positive likelihoods, but the advertised island-crossing property needs either a rigorous proof under explicit assumptions or a weakening of the claim.
  3. [§2.11.1, Eq. (32); §3.2–§3.4] The neural-network experiments sample with respect to an ad hoc likelihood L(x)=exp(-f_loss(x) D_eff/(2 Δf_loss)), where D_eff and Δf_loss are user-chosen. This is not derived from a Bayesian model and is acknowledged as heuristic in the text. The abstract and conclusions nevertheless state that the method 'sample[s] the posterior distributions of neural network outputs' and conclude that it 'provides fair sampling.' These statements go beyond what Eq. (32) defines. Either the manuscript should explicitly state that the neural-network experiments target a user-specified loss-tolerance distribution and consistently use 'approximate' language, or it should provide a principled calibration for L(x) that connects D_eff and Δf_loss to a well-defined posterior.
minor comments (5)
  1. [Eq. (17)] The symbol L is used for both the likelihood and the radiance; the notation becomes confusing, especially in Eq. (17) where the two quantities appear together. Please use a distinct symbol (e.g., script L or R) for radiance.
  2. [Algorithm 1] The pseudocode reassigns Δs to √D Δs inside the procedure. Since Δs is also the function argument, this is a source of confusion; use a separate variable for the scaled step size or state explicitly that the scaling is applied once at initialization.
  3. [§2.5 / Appendix A] The blackbody-radiator boundary treatment is described in the text and appendix but is not included in Algorithm 1. If it is intended to be part of the proposed sampling method, a separate pseudocode block would help. If it is only an extension, the abstract's 'including holes' claim should be qualified.
  4. [§2.9.2] The claimed equivalence between Goodman–Weare stretch moves and a limiting ray-tracing cone is sketched informally. Since this is one of the paper's stated special cases, a short derivation of the acceptance rule from the ray-tracing formalism would significantly strengthen the unification claim.
  5. [Abstract / §2.9.1] The abstract mentions 'extending past work on isokinetic sampling,' but the text does not explain or cite this connection. Please add a reference or remove the phrase.

Circularity Check

0 steps flagged

No significant circularity: the central ray-tracing derivation is self-contained, and self-citations are not load-bearing.

full rationale

The core claim is that choosing n(x)=L(x)^{1/(D-1)} makes the invariant étendue/radiance measure proportional to the target likelihood. This is a design choice, not a circular prediction: Eqs. (2)-(6) prove conservation of basic radiance from Snell's law, and Eqs. (12)-(17) then show that this choice makes exact ray paths satisfy detailed balance and provides a Metropolis correction for imperfect integrators. The acceptance criterion in Eq. (17) compares the computed radiance boost to the target-likelihood ratio, so it is a genuine check rather than an input assumed to be true. The generalized weighting version, Eqs. (18)-(19), follows from the same invariant. No fitted parameters enter the stationarity proof. Section 2.9's claim that HMC, Metropolis, Gibbs, and Monte Carlo integration are special cases is a reinterpretation of existing samplers in the ray-tracing language; it is not used as evidence for the sampler's validity and does not reduce the central claim to prior results. The self-citations in the paper (Behroozi et al. 2019 for training data, and 'Behroozi et al., in prep.' for a future direction) are not load-bearing for the derivation. The ad hoc likelihood scaling in Eq. (32) is a heuristic for the neural-network applications, but it does not enter the proof of fair sampling. The ergodicity argument in Section 2.5 has a correctness gap for zero-likelihood holes, because the ray equation degenerates where n=0, but an unsupported overclaim is a soundness concern, not circularity: the failure is not that the conclusion is assumed as an input. Overall, the paper's derivation chain does not exhibit a reduction of any prediction to its own inputs.

Axiom & Free-Parameter Ledger

4 free parameters · 5 axioms · 0 invented entities

The method rests on standard radiative-transfer conservation laws plus a handful of user-chosen hyperparameters (D_eff, Δf_loss, step size, refresh rate) that define the posterior for the NN applications. No new physical entities are introduced.

free parameters (4)
  • Effective parameter count D_eff = 42 (MLP), 1.4e4 (ResNet), assumed 1.5e9 (GPT-2)
    Introduced in Eq. 32 to convert a loss tolerance into a likelihood; chosen by trial and error to match desired loss width.
  • Loss tolerance Δf_loss = 0.0042, 0.18, 0.1875
    User-chosen posterior width; arbitrary.
  • Step size Δs = hand-tuned per application (e.g., Δt=2e-4 for MLP, 2e-6 for ResNet, 5e-6 for GPT-2)
    Chosen to balance acceptance rate and exploration; no automatic tuning.
  • Momentum refresh rate f = 5Δt in MLP; others not specified
    Hand-tuned.
axioms (5)
  • standard math Ray propagation follows the ray equation d/ds(n dx/ds)=∇n (Eq. 20)
    Adopted from Hamiltonian optics (Born & Wolf).
  • standard math Conservation of étendue in arbitrary dimension D (Eq. 8)
    Derived in Sections 2.2-2.3; relies on the ray-bundle picture and smoothness of n.
  • domain assumption Likelihood L(x) is smooth, positive, and differentiable wherever sampled
    Needed for the gradients and for n>0; zero-likelihood holes are treated separately via blackbody radiators.
  • ad hoc to paper The discrete integrator (Eq. 23) is reversible and its solid-angle Jacobian is exactly (sinθ_f/sinθ_i)^{D-1}
    The paper states this implicitly through the radiance-boost acceptance criterion but does not prove it for the discrete map.
  • ad hoc to paper For neural networks, the constructed likelihood (Eq. 32) is an acceptable substitute for the unknown true posterior
    Used to define a posterior over network outputs within a loss tolerance; D_eff is chosen post hoc.

pith-pipeline@v1.3.0-alltime-deepseek · 41453 in / 21720 out tokens · 209872 ms · 2026-08-04T07:25:27.631309+00:00 · methodology

0 comments
read the original abstract

We derive a family of Markov Chain Monte Carlo (MCMC) sampling methods based on following ray paths in a medium where the refractive index $n(x)$ is a function of the desired likelihood $\mathcal{L}(x)$, extending past work on isokinetic sampling. The simplest ray tracing method propagates rays at constant speed through parameter space, leading to orders of magnitude higher resilience to heating for stochastic gradients as compared to both Hamiltonian Monte Carlo (HMC) and Stochastic Gradient HMC, as well as the ability to cross any likelihood barrier, including holes in parameter space. Using the simplest ray tracing method, we sample the posterior distributions of neural network outputs for a variety of different architectures, including a preliminary exploration of the 1.5 billion-parameter GPT-2 (Generative Pre-trained Transformer 2) architecture, all on a single consumer-level GPU. We also show that prior samplers including traditional HMC, the original microcanonical HMC, Metropolis, Gibbs, and even Monte Carlo integration are special cases within a generalized ray tracing framework, which can sample according to an arbitrary weighting function. Public code and documentation for C, JAX, and PyTorch are available at https://bitbucket.org/pbehroozi/ray-tracing-sampler/src

Figures

Figures reproduced from arXiv: 2510.25824 by Peter Behroozi.

Figure 1
Figure 1. Figure 1: — Example sampling dynamics for ray tracing and Hamiltonian Monte Carlo (HMC) for a 2D Gaussian mixture, with L (x) = [exp(−0.5(𝑥1 − 2) 2 ) + exp(−0.5(𝑥1 + 2) 2 ) ] exp(−0.5𝑥 2 2 ). For this case, as well as any other 2D distribution, fair sampling with ray tracing occurs when the spatially-varying refractive index is the same as the likelihood function (i.e., 𝑛(x) = L (x)). This figure illustrates the ben… view at source ↗
Figure 3
Figure 3. Figure 3: — Parameters describing rays propagating from a differential emission area 𝑑𝐴𝐸 to a differential receiving area 𝑑𝐴𝑅. The angles between the rays and the normal vectors of the emitter and receiver are 𝜃𝐸 and 𝜃𝑅, respectively. The differential solid angle encompassed by the rays at the emitter is 𝑑Ω𝐸, and that encompassed by the rays at the receiver is 𝑑Ω𝑅. As discussed in the text, the étendue at the emitte… view at source ↗
Figure 2
Figure 2. Figure 2: — Refraction of light for an arbitrary dimensionality 𝐷. This figure shows an incoming beam of light with radiance 𝐿1 in a medium with refractive index 𝑛1, crossing a differential area 𝑑𝐴at the interface with another medium with refractive index 𝑛2, resulting in an outgoing beam of light with radiance 𝐿2. By Snell’s law, 𝑛1 sin 𝜃1 = 𝑛2 sin 𝜃2, where 𝜃1 and 𝜃2 are the respective angles to the interface norm… view at source ↗
Figure 4
Figure 4. Figure 4: — Path dynamics of Hamiltonian Monte Carlo. A particle moving from potential 𝑈1 to 𝑈2 experiences a force normal to the potential boundary, causing it to accelerate. The velocity component perpendicular to the force, v⊥, is unaffected. Hence, we have the geometrical identity |v1 | sin 𝜃1 = |v⊥ | = |v2 | sin 𝜃2, and the path is identical to that of a light ray traveling in a medium with refractive index 𝑛 =… view at source ↗
Figure 5
Figure 5. Figure 5: — This figure shows ray tracing samples for a 10,000–dimensional Gaussian likelihood (ln L = −0.5|x| 2 ), using a moderate step size equivalent to a change in propagation direction of ⟨Δ𝜙⟩ = 0.25 at each step. The non-Metropolis cases both converge to stable distributions that are biased, so a Metropolis test can be helpful to achieve larger step sizes when exact likelihoods are used. -1.5 -1 -0.5 0 0.5 1 … view at source ↗
Figure 6
Figure 6. Figure 6: — Leapfrog path (Δ𝜙 = 0.8) vs. true path for the potential 𝑈 = 𝑥 2 . Kick-drift-kick integrators sample the red points, which are typically outside the true path, whereas drift-kick-drift integrators sample the blue points, which are typically inside the true path. At in-between locations, the path radii are correct, but the velocities are not tangent to the true path. Randomly choosing between KDK and DKD… view at source ↗
Figure 7
Figure 7. Figure 7: — Effective sample sizes (ESS) per gradient evaluation for kick-drift￾kick (KDK), drift-kick-drift (DKD), Omelyan et al. (2003) Minimal Norm second-order (Minimal Norm), and Forest & Ruth (1990)/Yoshida (1990) integrators, all for the same 10,000–dimensional Gaussian likelihood. All show a characteristic sharp dropoff with increasing step size (shown in units of the average angular change Δ𝜙). We find that… view at source ↗
Figure 8
Figure 8. Figure 8: — True likelihood distribution for the ray tracing algorithm using stochastic gradients (Eq. 34) with a 10,000-dimensional Gaussian distribu￾tion and a step size of Δ𝜙 = 0.03. Minimal deviation from the true distribu￾tion (thick sand-colored line) is observed until 𝜎sto = 10, corresponding to perturbing the center of the Gaussian by an amount 10 times larger than its standard deviation(!) By comparison, Ha… view at source ↗
Figure 9
Figure 9. Figure 9: — Comparison between the resilience of ray tracing and traditional Hamiltonian Monte Carlo (HMC) to noise in stochastic gradients, again for a 10,000-dimensional Gaussian distribution. The vertical axis shows the deviation in the median log likelihood (ln L) as a function of the gradient noise (parameterized by 𝜎sto in Eq. 34). At a moderate step size of Δ𝜙 = 0.03, ray tracing can tolerate 16× larger 𝜎sto … view at source ↗
Figure 10
Figure 10. Figure 10: — Left: A 3D projection of a sample trajectory in weight space from ray tracing in a 1433-dimensional neural network (a multi-layer perceptron) in Section 3.2; dimensions 5, 15, and 150 are shown. The path resembles a tangled piece of spaghetti, but the path never crosses itself, due to the many dimensions available for exploration. Frequent twists and turns are evident, indicating that the likelihood spa… view at source ↗
Figure 11
Figure 11. Figure 11: — Top left: distributions of the loss function (mean-squared error; MSE) on the validation set for the multi-layer perceptron application in Section 3.2. Adam has the lowest loss, as it searches for loss minima. Ray tracing and HMC both sample from the posterior distribution of models within a given tolerance of the minimum loss (here chosen to be Δ 𝑓loss = 0.0042, or ∼ 10% of typical MSE), and so have hi… view at source ↗
Figure 12
Figure 12. Figure 12: — Top left: Root mean square error (RMSE) for training vs. validation data, using ray tracing on a ResNet-34 network with 22 million free parameters and a training data input size of ∼12,000 (Section 3.3). Even after sampling for >500,000 epochs, training and validation loss are similar, suggesting resilience to overfitting. The grey shaded region shows the 68% range of Adam predictions after 10 epochs of… view at source ↗
Figure 13
Figure 13. Figure 13: — Left: The distribution of standardized differences (i.e., difference in predictions divided by the posterior standard deviation) of galaxies in the validation set for 1) a model generated using 5000 epochs of ray tracing on an identically-sized but non-overlapping training set drawn from the same data distribution, 2) a model generated using 5000 epochs of ray tracing using the same training set but a d… view at source ↗
Figure 14
Figure 14. Figure 14: — Examples of galaxies with the shortest short (top row) autocorrelation times (<100 epochs) and the longest (bottom row) autocorrelation times (>105 epochs). Galaxies with long autocorrelation times are typically outliers in some sense: they often have unusual colors, mergers, or contamination with stars, leading to unconstrained metallicities. Colors shown are from SDSS gri-band images. 1×105 2×105 3×10… view at source ↗
Figure 15
Figure 15. Figure 15: — The sequence of predicted metallicities for a galaxy in the val￾idation set (shown in the center of the inset) with an exceptionally long au￾tocorrelation time. In practice, the metallicity of the galaxy is unconstrained given the training set and the model architecture, and so the samples do not converge to any fixed distribution over the fraction of the posterior set that has been sampled. Of note, th… view at source ↗
Figure 16
Figure 16. Figure 16: — Left: Validation loss (cross-entropy) as a function of total tokens processed. For reference, perplexity (i.e., the exponential of cross-entropy loss) is also shown. We chose a target loss (∼2.53; purple line) in between the official GPT-2 weights applied to our training set (red line) and the initial trained seed (blue line), corresponding to L (x) = 4 × 109 𝑓loss(x), and used a step size of 5 (corresp… view at source ↗
Figure 17
Figure 17. Figure 17: — Bayesian exploration of parameters for GPT-2 allows coloring validation text by both entropy and autocorrelation time. Blue: high entropy (> 4) but short autocorrelation (colored text is unpredictable given training set data); Purple: long autocorrelation (> 500M tokens) and low entropy (colored text is predictable, but the network struggles to learn the correct probability distribution); Bold Red: long… view at source ↗
Figure 18
Figure 18. Figure 18: — Autocorrelation time and entropy are uncorrelated across our validation dataset. This figure shows entropy vs. autocorrelation time for the most-probable next token, in units of billions of tokens; there are roughly 2×105 steps per billion tokens. More efficient network architectures than GPT￾2 would result in more efficient information flow into the network for a given amount of tokens processed, which… view at source ↗
Figure 19
Figure 19. Figure 19: — Bayesian sampling allows quantifying the uncertainty associated with a given input data vector, in this case, a query about a patient’s diagnosis given a list of symptoms (shown at the top of the figure). This figure shows the distribution of answers across different diagnoses (i.e., the GPT-2 Reported Diagnosis Probability, or 𝑃(diagnosis|model,symptoms)). Data points and error bars show the averages a… view at source ↗
Figure 20
Figure 20. Figure 20: — Comparison between sampling efficiencies (defined as gradients per effective sample) when perfect gradients are used, for ray tracing and HMC. No strong performance advantages are observed. The minimum dimension shown is 2, since ray tracing is not defined for 𝐷 = 1. Upper limits are shown for the Cauchy distribution, since the sampling efficiency as a function of step size and trajectory length had man… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

78 extracted references · 12 canonical work pages · 5 internal anchors

  1. [1]

    Abolfathi B., et al., 2018, @doi [ ] 10.3847/1538-4365/aa9e8a , https://ui.adsabs.harvard.edu/abs/2018ApJS..235...42A 235, 42

  2. [2]

    D., 2024, in 38th conference on Neural Information Processing Systems

    Agarwal S., \'Ciprijanovi\'c A., Nord B. D., 2024, in 38th conference on Neural Information Processing Systems . ( @eprint arXiv 2411.03334 )

  3. [3]

    Ahmad W., Simon E., Chithrananda S., Grand G., Ramsundar B., 2022, preprint, https://ui.adsabs.harvard.edu/abs/2022arXiv220901712A ( @eprint arXiv 2209.01712 )

  4. [4]

    N., Bates S., 2021, @doi [arXiv e-prints] 10.48550/arXiv.2107.07511 , https://ui.adsabs.harvard.edu/abs/2021arXiv210707511A p

    Angelopoulos A. N., Bates S., 2021, @doi [arXiv e-prints] 10.48550/arXiv.2107.07511 , https://ui.adsabs.harvard.edu/abs/2021arXiv210707511A p. arXiv:2107.07511

  5. [5]

    H., Hearin A

    Behroozi P., Wechsler R. H., Hearin A. P., Conroy C., 2019, @doi [ ] 10.1093/mnras/stz1182 , https://ui.adsabs.harvard.edu/abs/2019MNRAS.488.3143B 488, 3143

  6. [6]

    Gaussian Processes to speed up MCMC with automatic exploratory-exploitation effect

    Benavoli A., Wyse J., White A., 2021, @doi [arXiv e-prints] 10.48550/arXiv.2109.13891 , https://ui.adsabs.harvard.edu/abs/2021arXiv210913891B p. arXiv:2109.13891

  7. [7]

    Besag J., 1994, J. Roy. Statist. Soc. Ser. B, 56, 591

  8. [8]

    arXiv:1701.02434

    Betancourt M., 2017, @doi [arXiv e-prints] 10.48550/arXiv.1701.02434 , https://ui.adsabs.harvard.edu/abs/2017arXiv170102434B p. arXiv:1701.02434

  9. [9]

    arXiv:1607.03188

    Bierkens J., Fearnhead P., Roberts G., 2016, @doi [arXiv e-prints] 10.48550/arXiv.1607.03188 , https://ui.adsabs.harvard.edu/abs/2016arXiv160703188B p. arXiv:1607.03188

  10. [10]

    Pergamon Press

    Born M., Wolf E., 1980, Principles of optics: electromagnetic theory of propagation, interference and diffraction of light. Pergamon Press

  11. [11]

    J., Doucet A., 2015, @doi [arXiv e-prints] 10.48550/arXiv.1510.02451 , https://ui.adsabs.harvard.edu/abs/2015arXiv151002451B p

    Bouchard-C \^o t \'e A., Vollmer S. J., Doucet A., 2015, @doi [arXiv e-prints] 10.48550/arXiv.1510.02451 , https://ui.adsabs.harvard.edu/abs/2015arXiv151002451B p. arXiv:1510.02451

  12. [12]

    Stochastic Gradient Hamiltonian Monte Carlo

    Chen T., Fox E. B., Guestrin C., 2014, @doi [arXiv e-prints] 10.48550/arXiv.1402.4102 , https://ui.adsabs.harvard.edu/abs/2014arXiv1402.4102C p. arXiv:1402.4102

  13. [13]

    Choudhary K., et al., 2022, @doi [npj Computational Materials] 10.1038/s41524-022-00734-6 , 8, 59

  14. [14]

    arXiv:2204.02311

    Chowdhery A., et al., 2022, @doi [arXiv e-prints] 10.48550/arXiv.2204.02311 , https://ui.adsabs.harvard.edu/abs/2022arXiv220402311C p. arXiv:2204.02311

  15. [15]

    N., Wild S

    \'Ciprijanovi\'c A., Lewis A., Pedro K., Madireddy S., Nord B., Perdue G. N., Wild S. M., 2023, @doi [Mach. Learn. Sci. Tech.] 10.1088/2632-2153/acca5f , 4, 025013

  16. [16]

    Corbella A., Spencer S. E. F., Roberts G. O., 2022, @doi [arXiv e-prints] 10.48550/arXiv.2206.11410 , https://ui.adsabs.harvard.edu/abs/2022arXiv220611410C p. arXiv:2206.11410

  17. [17]

    arXiv:1410.8516

    Dinh L., Krueger D., Bengio Y., 2014, @doi [arXiv e-prints] 10.48550/arXiv.1410.8516 , https://ui.adsabs.harvard.edu/abs/2014arXiv1410.8516D p. arXiv:1410.8516

  18. [18]

    arXiv:2112.06905

    Du N., et al., 2021, @doi [arXiv e-prints] 10.48550/arXiv.2112.06905 , https://ui.adsabs.harvard.edu/abs/2021arXiv211206905D p. arXiv:2112.06905

  19. [19]

    D., Pendleton B

    Duane S., Kennedy A. D., Pendleton B. J., Roweth D., 1987, @doi [Physics Letters B] 10.1016/0370-2693(87)91197-X , https://ui.adsabs.harvard.edu/abs/1987PhLB..195..216D 195, 216

  20. [20]

    J., Deem M

    Earl D. J., Deem M. W., 2005, @doi [Phys. Chem. Chem. Phys.] 10.1039/B509983H , 7, 3910

  21. [21]

    P., 2008, @doi [ ] 10.1111/j.1365-2966.2007.12353.x , https://ui.adsabs.harvard.edu/abs/2008MNRAS.384..449F 384, 449

    Feroz F., Hobson M. P., 2008, @doi [ ] 10.1111/j.1365-2966.2007.12353.x , https://ui.adsabs.harvard.edu/abs/2008MNRAS.384..449F 384, 449

  22. [22]

    P., Cameron E., Pettitt A

    Feroz F., Hobson M. P., Cameron E., Pettitt A. N., 2019, @doi [The Open Journal of Astrophysics] 10.21105/astro.1306.2144 , https://ui.adsabs.harvard.edu/abs/2019OJAp....2E..10F 2, 10

  23. [23]

    W., Lang D., Goodman J., 2013, @doi [ ] 10.1086/670067 , http://adsabs.harvard.edu/abs/2013PASP..125..306F 125, 306

    Foreman-Mackey D., Hogg D. W., Lang D., Goodman J., 2013, @doi [ ] 10.1086/670067 , http://adsabs.harvard.edu/abs/2013PASP..125..306F 125, 306

  24. [24]

    D., 1990, @doi [Physica D: Nonlinear Phenomena] https://doi.org/10.1016/0167-2789(90)90019-L , 43, 105

    Forest E., Ruth R. D., 1990, @doi [Physica D: Nonlinear Phenomena] https://doi.org/10.1016/0167-2789(90)90019-L , 43, 105

  25. [25]

    arXiv:2409.20252

    Fouesneau M., et al., 2024, @doi [arXiv e-prints] 10.48550/arXiv.2409.20252 , https://ui.adsabs.harvard.edu/abs/2024arXiv240920252F p. arXiv:2409.20252

  26. [26]

    Geman S., Geman D., 1984, @doi [IEEE Transactions on Pattern Analysis and Machine Intelligence] 10.1109/TPAMI.1984.4767596 , PAMI-6, 721

  27. [27]

    Girolami M., Calderhead B., 2011, @doi [Journal of the Royal Statistical Society: Series B (Statistical Methodology)] https://doi.org/10.1111/j.1467-9868.2010.00765.x , 73, 123

  28. [28]

    Goodman J., Weare J., 2010, @doi [Communications in Applied Mathematics and Computational Science] 10.2140/camcos.2010.5.65 , http://adsabs.harvard.edu/abs/2010CAMCS...5...65G 5, 65

  29. [29]

    Haario H., Saksman E., Tamminen J., 2001, Bernoulli, 7, pp. 223

  30. [30]

    K., 1970, @doi [Biometrika] 10.1093/biomet/57.1.97 , https://ui.adsabs.harvard.edu/abs/1970Bimka..57...97H 57, 97

    Hastings W. K., 1970, @doi [Biometrika] 10.1093/biomet/57.1.97 , https://ui.adsabs.harvard.edu/abs/1970Bimka..57...97H 57, 97

  31. [31]

    arXiv:1502.01852

    He K., Zhang X., Ren S., Sun J., 2015, @doi [arXiv e-prints] 10.48550/arXiv.1502.01852 , https://ui.adsabs.harvard.edu/abs/2015arXiv150201852H p. arXiv:1502.01852

  32. [32]

    He K., Zhang X., Ren S., Sun J., 2016, in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR. p. 1 ( @eprint arXiv 1512.03385 ), @doi 10.1109/CVPR.2016.90

  33. [33]

    D., Wilson A

    Izmailov P., Vikram S., Hoffman M. D., Wilson A. G., 2021, @doi [arXiv e-prints] 10.48550/arXiv.2104.14421 , https://ui.adsabs.harvard.edu/abs/2021arXiv210414421I p. arXiv:2104.14421

  34. [34]

    arXiv:1505.05770

    Jimenez Rezende D., Mohamed S., 2015, @doi [arXiv e-prints] 10.48550/arXiv.1505.05770 , https://ui.adsabs.harvard.edu/abs/2015arXiv150505770J p. arXiv:1505.05770

  35. [35]

    P., Ba J., 2014, arXiv e-prints, https://arxiv.org/abs/1412.6980 p

    Kingma D. P., Ba J., 2014, arXiv e-prints, https://arxiv.org/abs/1412.6980 p. arXiv:1412.6980

  36. [36]

    arXiv:1706.02515

    Klambauer G., Unterthiner T., Mayr A., Hochreiter S., 2017, @doi [arXiv e-prints] 10.48550/arXiv.1706.02515 , https://ui.adsabs.harvard.edu/abs/2017arXiv170602515K p. arXiv:1706.02515

  37. [37]

    Klypin A., Yepes G., Gottl \"o ber S., Prada F., He S., 2016, @doi [ ] 10.1093/mnras/stw248 , http://adsabs.harvard.edu/abs/2016MNRAS.457.4340K 457, 4340

  38. [38]

    Krashen S., 2000, Educators for urban minorities, https://eric.ed.gov/?id=EJ631864 1, 53

  39. [39]

    arXiv:1612.01474

    Lakshminarayanan B., Pritzel A., Blundell C., 2016, @doi [arXiv e-prints] 10.48550/arXiv.1612.01474 , https://ui.adsabs.harvard.edu/abs/2016arXiv161201474L p. arXiv:1612.01474

  40. [40]

    pp 4188--4188, @doi 10.1109/PIERS.2016.7735574

    Li C., Sun Y., Liu J., 2016, in 2016 Progress in Electromagnetic Research Symposium (PIERS). pp 4188--4188, @doi 10.1109/PIERS.2016.7735574

  41. [41]

    Lin Z., et al., 2023, @doi [Science] 10.1126/science.ade2574 , 379, 1123

  42. [42]

    arXiv:2501.17917

    Loaiza-Ganem G., Villecroze V., Wang Y., 2025, @doi [arXiv e-prints] 10.48550/arXiv.2501.17917 , https://ui.adsabs.harvard.edu/abs/2025arXiv250117917L p. arXiv:2501.17917

  43. [43]

    V., 2023, @doi [Appl

    McKeon B., Goncharov A. V., 2023, @doi [Appl. Opt.] 10.1364/AO.501102 , 62, 8621

  44. [44]

    Melchior P., Joseph R., Sanchez J., MacCrann N., Gruen D., 2021, @doi [Nature Reviews Physics] 10.1038/s42254-021-00353-y , 3, 712

  45. [45]

    W., Rosenbluth M

    Metropolis N., Rosenbluth A. W., Rosenbluth M. N., Teller A. H., Teller E., 1953, @doi [ ] 10.1063/1.1699114 , https://ui.adsabs.harvard.edu/abs/1953JChPh..21.1087M 21, 1087

  46. [46]

    M., 2012, arXiv e-prints, https://ui.adsabs.harvard.edu/abs/2012arXiv1206.1901N p

    Neal R. M., 2012, arXiv e-prints, https://ui.adsabs.harvard.edu/abs/2012arXiv1206.1901N p. arXiv:1206.1901

  47. [47]

    Ohno H., 2020, @doi [J. Opt. Soc. Am. A] 10.1364/JOSAA.378829 , 37, 411

  48. [48]

    Omelyan I., Mryglod I., Folk R., 2003, @doi [Computer Physics Communications] https://doi.org/10.1016/S0010-4655(02)00754-3 , 151, 272

  49. [49]

    arXiv:2303.08774

    OpenAI 2023, @doi [arXiv e-prints] 10.48550/arXiv.2303.08774 , https://ui.adsabs.harvard.edu/abs/2023arXiv230308774O p. arXiv:2303.08774

  50. [50]

    Parzen E., 1962, @doi [The Annals of Mathematical Statistics] 10.1214/aoms/1177704472 , 33, 1065

  51. [51]

    arXiv:2406.17557

    Penedo G., Kydl \' c ek H., Ben allal L., Lozhkov A., Mitchell M., Raffel C., Von Werra L., Wolf T., 2024, @doi [arXiv e-prints] 10.48550/arXiv.2406.17557 , https://ui.adsabs.harvard.edu/abs/2024arXiv240617557P p. arXiv:2406.17557

  52. [52]

    Peters E. A. J. F., de With G., 2012, @doi [ ] 10.1103/PhysRevE.85.026703 , https://ui.adsabs.harvard.edu/abs/2012PhRvE..85b6703P 85, 026703

  53. [53]

    Puzyrev V., Salles T., Surma G., Elders C., 2022, @doi [Geoscience Letters] 10.1186/s40562-022-00241-y , 9, 32

  54. [54]

    Radford A., Wu J., Child R., Luan D., Amodei D., Sutskever I., et al., 2019, OpenAI blog, 1, 9

  55. [55]

    O., Tweedie R

    Roberts G. O., Tweedie R. L., 1996, Bernoulli, 2, 341

  56. [56]

    B., Silverstein E., Seljak U., 2022, @doi [arXiv e-prints] 10.48550/arXiv.2212.08549 , https://ui.adsabs.harvard.edu/abs/2022arXiv221208549R p

    Robnik J., De Luca G. B., Silverstein E., Seljak U., 2022, @doi [arXiv e-prints] 10.48550/arXiv.2212.08549 , https://ui.adsabs.harvard.edu/abs/2022arXiv221208549R p. arXiv:2212.08549

  57. [57]

    Metropolis Adjusted Microcanonical Hamiltonian Monte Carlo

    Robnik J., Cohn-Gordon R., Seljak U., 2025, @doi [arXiv e-prints] 10.48550/arXiv.2503.01707 , https://ui.adsabs.harvard.edu/abs/2025arXiv250301707R p. arXiv:2503.01707

  58. [58]

    Rodr \' guez-Puebla A., Behroozi P., Primack J., Klypin A., Lee C., Hellinger D., 2016, @doi [ ] 10.1093/mnras/stw1705 , http://adsabs.harvard.edu/abs/2016MNRAS.462..893R 462, 893

  59. [59]

    ( @eprint arXiv 2311.01588 )

    Roncoli A., \'Ciprijanovi\'c A., Voetberg M., Villaescusa-Navarro F., Nord B., 2023, in 37th Conference on Neural Information Processing Systems . ( @eprint arXiv 2311.01588 )

  60. [60]

    Rosenblatt M., 1956, @doi [The Annals of Mathematical Statistics] 10.1214/aoms/1177728190 , 27, 832

  61. [61]

    R., 2003, in Erbacher R

    Satoh T. R., 2003, in Erbacher R. F., Chen P. C., Roberts J. C., Groehn M. T., Borner K., eds, Society of Photo-Optical Instrumentation Engineers (SPIE) Conference Series Vol. 5009, Visualization and Data Analysis 2003. pp 277--285, @doi 10.1117/12.473932

  62. [62]

    V., eds, American Institute of Physics Conference Series Vol

    Skilling J., 2004, in Fischer R., Preuss R., Toussaint U. V., eds, American Institute of Physics Conference Series Vol. 735, Bayesian Inference and Maximum Entropy Methods in Science and Engineering: 24th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering. pp 395--405, @doi 10.1063/1.1835238

  63. [63]

    D., 1996

    Sokal A. D., 1996. https://api.semanticscholar.org/CorpusID:14817657

  64. [64]

    S., 2020, @doi [ ] 10.1093/mnras/staa278 , https://ui.adsabs.harvard.edu/abs/2020MNRAS.493.3132S 493, 3132

    Speagle J. S., 2020, @doi [ ] 10.1093/mnras/staa278 , https://ui.adsabs.harvard.edu/abs/2020MNRAS.493.3132S 493, 3132

  65. [65]

    Towards Training Billion Parameter Graph Neural Networks for Atomic Simulations

    Sriram A., Das A., Wood B. M., Goyal S., Zitnick C. L., 2022, @doi [arXiv e-prints] 10.48550/arXiv.2203.09697 , https://ui.adsabs.harvard.edu/abs/2022arXiv220309697S p. arXiv:2203.09697

  66. [66]

    G., Vanden-Eijnden E., 2010, Communications in Mathematical Sciences, 8, 217

    Tabak E. G., Vanden-Eijnden E., 2010, Communications in Mathematical Sciences, 8, 217

  67. [67]

    arXiv:2506.06921

    Ting Y.-S., O'Briain T., 2025, @doi [arXiv e-prints] 10.48550/arXiv.2506.06921 , https://ui.adsabs.harvard.edu/abs/2025arXiv250606921T p. arXiv:2506.06921

  68. [68]

    K., 2001, Scandinavian Journal of Statistics, 28, 205

    Tjelmeland H., Hegstad B. K., 2001, Scandinavian Journal of Statistics, 28, 205

  69. [69]

    N., Kaiser L., Polosukhin I., 2017, @doi [arXiv e-prints] 10.48550/arXiv.1706.03762 , https://ui.adsabs.harvard.edu/abs/2017arXiv170603762V p

    Vaswani A., Shazeer N., Parmar N., Uszkoreit J., Jones L., Gomez A. N., Kaiser L., Polosukhin I., 2017, @doi [arXiv e-prints] 10.48550/arXiv.1706.03762 , https://ui.adsabs.harvard.edu/abs/2017arXiv170603762V p. arXiv:1706.03762

  70. [70]

    Vogelsberger M., Marinacci F., Torrey P., Puchwein E., 2020, @doi [Nature Reviews Physics] 10.1038/s42254-019-0127-2 , https://ui.adsabs.harvard.edu/abs/2020NatRP...2...42V 2, 42

  71. [71]

    arXiv:2002.02405

    Wenzel F., et al., 2020, @doi [arXiv e-prints] 10.48550/arXiv.2002.02405 , https://ui.adsabs.harvard.edu/abs/2020arXiv200202405W p. arXiv:2002.02405

  72. [72]

    Wong K. W. K., Gabri \'e M., Foreman-Mackey D., 2023, @doi [The Journal of Open Source Software] 10.21105/joss.05021 , https://ui.adsabs.harvard.edu/abs/2023JOSS....8.5021W 8, 5021

  73. [73]

    F., Boada S., 2019, @doi [ ] 10.1093/mnras/stz333 , https://ui.adsabs.harvard.edu/abs/2019MNRAS.484.4683W 484, 4683

    Wu J. F., Boada S., 2019, @doi [ ] 10.1093/mnras/stz333 , https://ui.adsabs.harvard.edu/abs/2019MNRAS.484.4683W 484, 4683

  74. [74]

    Yoshida H., 1990, @doi [Physics Letters A] https://doi.org/10.1016/0375-9601(90)90092-3 , 150, 262

  75. [75]

    I., 2019, in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

    You K., Long M., Cao Z., Wang J., Jordan M. I., 2019, in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp 2720--2729

  76. [76]

    T., Wang W., Bai J., Wang Z., Song Y., 2025, @doi [arXiv e-prints] 10.48550/arXiv.2505.13259 , https://ui.adsabs.harvard.edu/abs/2025arXiv250513259Z p

    Zheng T., Deng Z., Tsang H. T., Wang W., Bai J., Wang Z., Song Y., 2025, @doi [arXiv e-prints] 10.48550/arXiv.2505.13259 , https://ui.adsabs.harvard.edu/abs/2025arXiv250513259Z p. arXiv:2505.13259

  77. [77]

    Zhou S., et al., 2025, npj Artificial Intelligence, 1, 9

  78. [78]

    write newline

    " write newline "" before.all 'output.state := FUNCTION fin.entry write newline FUNCTION new.block output.state before.all = 'skip after.block 'output.state := if FUNCTION new.sentence output.state after.block = 'skip output.state before.all = 'skip after.sentence 'output.state := if if FUNCTION not #0 #1 if FUNCTION and 'skip pop #0 if FUNCTION or pop #1...