Pith. sign in

REVIEW 3 major objections 4 minor 118 references

Nonlocal Monte Carlo via Reinforcement Learning

T0 review · 3 major / 4 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Reinforcement learning learns nonlocal cluster moves purely from energy signals, beating simulated annealing on hard 4-SAT.

desk verdict RLNMC replaces the handcrafted backbone threshold in nonlocal Monte Carlo with a PPO-trained GNN/GRU policy, and on scale-free 4-SAT it clearly beats both SA and NMC; but the abstract's claim of improvement on uniform-random 4-SAT across all three metrics is not supported by what the paper actually reports. read the letter →

arxiv 2508.10520 v1 pith:BS4MS37L submitted 2025-08-14 cs.LG cond-mat.dis-nn

Dmitrii Dobrynin , Masoud Mohseni , John Paul Strachan This is my paper · ORCID
classification cs.LGcond-mat.dis-nn
keywords reinforcementlearningnonlocalMonteCarlocombinatorialoptimization4-SATsimulatedannealinggraphneuralnetworkbackbonevariablessolutiondiversity
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Simulated annealing on hard 4-SAT stalls because variables that stay frozen inside a basin of attraction cannot be flipped one spin at a time. This paper shows that a reinforcement-learned policy can learn which variables to flip together as a nonlocal backbone move, replacing the hand-tuned threshold rule of the Nonequilibrium Nonlocal Monte Carlo (NMC) method. The policy is trained without labels or demonstrations, using only energy improvements as rewards and per-variable local-field magnitudes at local minima as states. On scale-free 4-SAT it roughly halves the time-to-solution of simulated annealing, on uniform random 4-SAT it matches or beats NMC, and it continues to improve at problem sizes four times larger than those used in training without any retuning. The paper's conclusion is that RL can discover useful nonlocal jump strategies purely from energy signals.

What carries the argument

The load-bearing mechanism is the NMC backbone jump: variables with large make\u2013break fields $H_i \equiv [E(x_i \to \bar{x}_i) - E(x_i)]/2$ are treated as rigid backbones of the current basin, randomized as a cluster, and then re-equilibrated at low temperature with non-backbone variables, producing a transition between basins that single-spin MCMC cannot make. RLNMC replaces the threshold rule $|H_i| \ge r$ with a trainable policy: a factor-graph self-attention GNN with per-variable and global GRU memories maps the local-minimum state, the $|H_i|$ values, the best energy, and the temperature to a Bernoulli backbone probability per variable, trained by PPO with the energy-improvement rew

What would settle it

Ablate the state representation: train the same RLNMC pipeline on the same $N = 500$ uniform random 4-SAT instances with the $|H_i|$ inputs permuted or zeroed, keeping the spin-state inputs; if the time-to-solution and residual-energy gains over threshold-based NMC survive, the local-field geometry is not the information doing the work. Alternatively, run the released implementations on the same 384 instances per class and check whether the trained policy's backbone selections track known planted backbone sets better than the $|H_i| \ge r$ rule.

Watch

Extended reading notes

Core claim

RLNMC adopts the three-stage NMC transition: at a local minimum, a set of backbone variables is randomized while the others are swept at low temperature with the backbones fixed, after which full Monte Carlo sweeps settle into a new basin. Where NMC decides the backbone set by thresholding the absolute local fields $|H_i|$, RLNMC uses a recurrent graph-neural-network policy that outputs a Bernoulli backbone probability for every variable. The RL state is the local-minimum spin configuration, the $|H_i|$ values, the best energy seen so far, and the current temperature; the reward is zero when the transition fails to beat the best energy seen so far, and otherwise equals the size of the improv

Load-bearing premise

The load-bearing premise is that a single local make\u2013break field magnitude $|H_i|$ per variable, read off at a local minimum, carries enough information for the policy to recognize good backbone clusters; if that signal is not sufficient for a new instance class, the claimed RL advantage is not guaranteed to transfer.

Editorial extensions

If this is right

  • Nonlocal move design can be learned rather than handcrafted: an RL policy trained only on energy improvements discovers backbone selections competitive with, and often better than, a tuned threshold rule.
  • The learned policy transfers across problem size: trained on uniform random 4-SAT at $N = 500$, RLNMC outperforms simulated annealing at $N = 1000$ and $N = 2000$ without retraining or additional hyperparameter tuning.
  • Nonlocal exploration improves solution diversity as a side effect: on the hardest scale-free instances RLNMC found several distinct solutions in cases where simulated annealing found none, even though diversity was not part of the reward.
  • The NMC framework can be grafted onto other base samplers; the paper explicitly proposes WalkSAT and parallel tempering as integrations, and notes that joint training of the annealing schedule and nonlocal moves is a future opportunity.
  • The learned jump schedules are more horizontal than NMC's: RLNMC makes distant moves with lower energy excitation, which the paper interprets as a more genuinely nonlocal strategy against freezing.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper leaves the belief-propagation correlation estimates of [18] out of its RL pipeline; a testable extension is to feed RLNMC localized correlations or magnetizations instead of $|H_i|$ and see whether transfer to the hardest instances improves, since the sufficiency of $|H_i|$ is asserted rather than proven.
  • The larger gain on scale-free instances suggests the policy may exploit the power-law degree structure, for example by treating hub variables as backbones; testing on scale-free instances with different exponents $b$, or on planted-backbone instances with a known frozen set, would reveal whether the learned strategy is structural or generic.
  • Because the paper optimized SA, then NMC, then RLNMC sequentially, the comparison likely understates both NMC and RLNMC; jointly optimizing the temperature schedule, jump frequency, and policy, as the outlook sketches, is a concrete way to test how much headroom remains.
  • The diversity improvement without a diversity reward hints that nonlocal cluster moves decorrelate independent replicas generically; an extension would add a diversity-seeking reward and measure the diversity integral over a wider approximation-ratio window.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes RLNMC, a reinforcement-learning variant of the Nonequilibrium Nonlocal Monte Carlo (NMC) solver. A recurrent GNN policy, trained with PPO using only energy-improvement rewards and per-variable local fields as state, replaces the threshold-based backbone selection heuristic of NMC. The authors evaluate RLNMC against MCMC simulated annealing and the NMC baseline on uniform random 4-SAT (N=500,1000,2000) and scale-free random 4-SAT (N=250), reporting residual energy, time-to-solution, and diversity of solutions. They claim that trained policies improve over both baselines on both benchmark families across all three metrics, and that RLNMC generalizes to sizes larger than those it was trained on without additional tuning.

Significance. If the headline claim is fully supported, the paper would be a meaningful demonstration that RL can discover nonlocal move strategies from energy signals alone and that the learned policies transfer across problem sizes. The study is methodologically careful in several respects: benchmarking is done on held-out instances disjoint from training, bootstrap resampling is used for error bars, the RL policy overhead is included in the time-to-solution estimate, and the code and trained models are released. However, the central claim as stated in the abstract is broader than the evidence presented: time-to-solution and diversity results appear only for the scale-free benchmark, and the residual-energy advantage over NMC on uniform random instances is described in the text as marginal once policy overhead is counted. The paper is therefore potentially valuable, but its main claim needs to be realigned with the reported data or supplemented with the missing uniform-random TTS and diversity experiments.

major comments (3)
  1. [Abstract; Sec. IV B] The abstract claims that the trained policies improve over MCMC SA and NMC on both uniform random and scale-free 4-SAT 'in terms of residual energy, time-to-solution, and diversity of solutions.' In the main text, TTS99 is reported only for scale-free N=250 (Fig. 4) and diversity only for scale-free N=250 (Fig. 7). For uniform random 4-SAT, only residual energy is reported (Figs. 5b, 6, 13); there is no TTS or diversity result. Moreover, Sec. IV B 2 states that at N=500 RLNMC 'slightly outperforms NMC at the same number of MC sweeps and matches NMC when the policy overhead is taken into account.' Thus the uniform-random TTS/diversity parts of the central claim are unsupported, and the residual-energy advantage over NMC at the trained size is marginal. The authors should either provide the missing uniform-random TTS and diversity measurements or revise the abstract and the concluding clai
  2. [Sec. IV B 2, Fig. 6] The generalization claim over NMC at N=1000 and N=2000 is weakened by the fact that the NMC baseline is not re-tuned for these sizes. The threshold r=3 was optimized on N=500 and then kept fixed, while the RLNMC policy also received no additional training at larger sizes. Since r is a free hyperparameter of NMC, the comparison at larger sizes may reflect a suboptimal fixed threshold rather than a genuine learned advantage. To make the claim 'RLNMC generalizes better than NMC' load-bearing, the authors should re-optimize r (or at least scan a few values) at each N and show that RLNMC still outperforms NMC; alternatively, the claim should be restricted to 'RLNMC transfers without retuning' and separated from a claim of superiority over a properly tuned NMC baseline.
  3. [App. A 3] The policy used for benchmarking was selected as the best of multiple training attempts, evaluated on the training instances (App. A 3: 'The best performing policy is used for benchmarking of instances not seen during training'). This model-selection protocol is not accounted for in the reported results. Because the scale-free TTS and diversity advantages are headline numbers, the authors should report the spread of held-out performance across training runs (e.g., mean, min, max) or fix the selection rule before benchmarking. Without this, the reported advantage could be an optimistic draw from multiple random training runs rather than a stable property of the method.
minor comments (4)
  1. [Sec. III C 1] Typo: 'Nonequlibrium' should be 'Nonequilibrium'.
  2. [App. A 3] The phrase 'discounted reward maximization of Eq. IV A' refers to a definition in Sec. IV A that is not numbered as Eq. IV A; please number the reward equation or fix the cross-reference.
  3. [Figs. 1 and 2] Figures 1 and 2 appear to have nearly identical captions and placeholder artwork. If they are intended to be distinct figures (one in the introduction and one in the NMC section), the captions and/or diagrams should be differentiated; if duplicated, one should be removed.
  4. [App. A 1 a] Typo: 'the the clause to variable ratio' should be 'the clause to variable ratio'.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: RLNMC is evaluated on held-out instances; self-citations define baselines/metrics but do not force the reported results.

full rationale

The central derivation is an empirical RL training loop: a policy for backbone selection is trained with PPO on 64 instances per problem class, and the reported metrics (residual energy, TTS99, diversity) are computed on the disjoint 320-instance benchmark sets (Sec. VI A, Sec. IV B). The reward r_t = 0 if E(s_{t+1}) - e_t > 0 else -(E(s_{t+1}) - e_t) is an energy-improvement signal, not the evaluation metric itself, and the policy is not fitted to the test-set TTS/diversity numbers. The NMC baseline is prior work by the authors [18,19], but it is used as a comparison target and subroutine, not as an unverified premise that forces the RLNMC conclusion; the RLNMC result is an empirical comparison on held-out instances. The diversity metric is defined following [19], another self-citation, but this is a definition, not a load-bearing uniqueness theorem. There is no equation-level reduction of the reported predictions to fitted parameters or to the training objective. The paper does contain an evidentiary gap: the abstract claims TTS and diversity improvements for both uniform random and scale-free benchmarks, while TTS (Fig. 4) and diversity (Fig. 7) are shown only for scale-free N=250, and Sec. IV B 2 concedes for uniform N=500 that 'RLNMC slightly outperforms NMC at the same number of MC sweeps and matches NMC when the policy overhead is taken into account.' This weakens the abstract's scope but is a missing-support issue, not circularity. Similarly, App. A 3's selection of the best policy on training instances ('The best performing policy is used for benchmarking of instances not seen during training') is standard model selection and does not make the held-out results tautological. Overall, no claim reduces by construction to its own input, so circularity score is 0.

Assumptions & free parameters 7 free parameters · 5 assumptions · 0 invented entities

The central result depends on a number of tuned hyperparameters, but most are shared across algorithms and therefore do not bias the comparison. The most load-bearing assumption is the sufficiency of local-field magnitudes as RL state, which is a domain assumption not independently verified. No new physical entities are introduced.

free parameters (7)
  • NMC threshold r = r=3 (uniform random), r=4.5 (scale-free)
    Tuned on 64 training instances to define backbone status for the NMC baseline; affects the comparison.
  • Initial and final inverse temperatures beta_i, beta_f = beta_i=3 (uniform), 2 (scale-free); beta_f=8
    Optimized for SA TTS99 on training instances; schedule shared by SA, NMC, RLNMC.
  • Nonlocal switch temperature beta_NMC = beta_NMC=5
    Estimated from TTS minimum; when NMC/RLNMC begin nonlocal moves.
  • Total sweeps Nsw.total = 3e4 (scale-free N=250); 5e4/1e5/2e5 (uniform N=500/1000/2000)
    Chosen runtime budget for each benchmark; N^2 scaling.
  • NMC move hyperparameters (Ncycles, NNMC steps, Nsw per move) = Ncycles=3; NNMC=53 (sf), 50 (uf); Nsw=100,200,400,800
    Optimized for NMC baseline; shared with RLNMC.
  • Energy scaling escale(N) = N/50
    Hand-picked normalization for RL state input, keeps energies at same order across sizes.
  • PPO hyperparameters = learning rate schedule, gamma=0.75, lambda_GAE=0.95, epsilon_clip=0.25, epochs=5, etc.
    Standard RL training configuration; affects training but not claimed as optimal.
assumptions (5)
  • domain assumption The random 4-SAT instances at alpha=9.884 (uniform) and alpha=9.2 (scale-free) are in the hard phase (rigidity/frozen) where local MCMC solvers struggle.
    Taken from prior literature [92] and [94]; if the benchmarks are not actually hard, the improvements are less significant.
  • domain assumption Local field magnitudes |H_i| are a sufficient signal for backbone selection.
    Stated in Sec. VI A 2 as a replacement for LBP magnetizations; the central RLNMC state input depends on this.
  • domain assumption The NMC transition procedure (randomize backbone variables, then low-temperature sweeps) is an effective nonlocal move.
    Assumed from [18]; RLNMC keeps this move structure and only learns the backbone selection.
  • standard math PPO with the given reward converges to a policy that improves energy.
    Standard RL convergence assumptions; not proven in the paper.
  • standard math The TTS99 estimation with beta distribution over replicas is a valid measure.
    From [96]; used to compute time-to-solution.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Nonlocal Monte Carlo via Reinforcement Learning." pith.science (2026). https://pith.science/paper/BS4MS37L

@misc{pith2026250810520,
  author       = {Pith},
  title        = {Pith review of: Nonlocal Monte Carlo via Reinforcement Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BS4MS37L}},
  note         = {Machine review of arXiv:2508.10520}
}
read the original abstract

Optimizing or sampling complex cost functions of combinatorial optimization problems is a longstanding challenge across disciplines and applications. When employing family of conventional algorithms based on Markov Chain Monte Carlo (MCMC) such as simulated annealing or parallel tempering, one assumes homogeneous (equilibrium) temperature profiles across input. This instance independent approach was shown to be ineffective for the hardest benchmarks near a computational phase transition when the so-called overlap-gap-property holds. In these regimes conventional MCMC struggles to unfreeze rigid variables, escape suboptimal basins of attraction, and sample high-quality and diverse solutions. In order to mitigate these challenges, Nonequilibrium Nonlocal Monte Carlo (NMC) algorithms were proposed that leverage inhomogeneous temperature profiles thereby accelerating exploration of the configuration space without compromising its exploitation. Here, we employ deep reinforcement learning (RL) to train the nonlocal transition policies of NMC which were previously designed phenomenologically. We demonstrate that the resulting solver can be trained solely by observing energy changes of the configuration space exploration as RL rewards and the local minimum energy landscape geometry as RL states. We further show that the trained policies improve upon the standard MCMC-based and nonlocal simulated annealing on hard uniform random and scale-free random 4-SAT benchmarks in terms of residual energy, time-to-solution, and diversity of solutions metrics.

Figures

Figures reproduced from arXiv: 2508.10520 by the authors.

Figure 1
Figure 1. FIG. 1. Energy landscape of a potentially hard problem fea [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. FIG. 2. Illustration of the NMC stages in [18]. 1) The “back [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. FIG. 3. Reinforcement Learning Nonlocal Monte Carlo. [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (13 more)
Figure 4
Figure 4. Figure 4: FIG. 4. Time-to-solution for the hardest instances of [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 6
Figure 6. Figure 6: FIG. 6. Comparison of heuristics at larger sizes. Hyperpa [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 7
Figure 7. Figure 7: FIG. 7. Diversity of configurations for scale-free random [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]
Figure 8
Figure 8. Figure 8: FIG. 8. Energy landscape exploration examples by trained [PITH_FULL_IMAGE:figures/full_fig_p009_8.png]
Figure 9
Figure 9. Figure 9: FIG. 9. Distance to the state with the best seen energy so far [PITH_FULL_IMAGE:figures/full_fig_p009_9.png]
Figure 10
Figure 10. Figure 10: FIG. 10. Recurrent policy architecture. It is incorporated into RLNMC as shown in Fig. 3. Exact description and hyperpa [PITH_FULL_IMAGE:figures/full_fig_p012_10.png]
Figure 11
Figure 11. Figure 11: FIG. 11. Distribution of variables [PITH_FULL_IMAGE:figures/full_fig_p016_11.png]
Figure 12
Figure 12. Figure 12: FIG. 12. Median and 10/90 percentiles of runtimes for scale [PITH_FULL_IMAGE:figures/full_fig_p016_12.png]
Figure 4
Figure 4. Figure 4: At first, TTS99 decreases, provided that some so￾lutions are being found. Later, TTS99 increases; i.e. new solutions are not being discovered quickly enough to jus￾tify the increased runtime τ of an algorithm. In this case, an independent restart is preferred with a sh…
Figure 14
Figure 14. Figure 14: FIG. 14. Average [PITH_FULL_IMAGE:figures/full_fig_p017_14.png]
Figure 4
Figure 4. Figure 4: In the case of the uniform random problem class, [PITH_FULL_IMAGE:figures/full_fig_p019_4.png]
Figure 16
Figure 16. Figure 16: FIG. 16. RLNMC policy inference overhead compared to [PITH_FULL_IMAGE:figures/full_fig_p019_16.png]
Figure 15
Figure 15. Figure 15: FIG. 15. Nonlocal nonequilibrium jump statistics for opti [PITH_FULL_IMAGE:figures/full_fig_p019_15.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

118 extracted references · 63 canonical work pages

  1. [1]

    make− break

    Nonlocal Simulated Annealing Following the intuition of Nonequlibrium Nonlocal Monte Carlo in [18] we propose a modified version of SA with integrated nonlocal moves (still calling it NMC for simplicity of notation). As will be shown below, stan- dard (MCMC) SA is relatively successful at quickly solv- ing/approximating the problems. However, it tends to ...

  2. [3]

    back- bone

    all spins MCMC Tlow Configuration space […,σb=1,…] […,σb=−1,…] sequence of NMC jumps GNN recurrent policy Save states , actions , and rewards at each step for GNN training with RL st at rt NMC step st−1 st st+1 backbone at−1 backbone at backbone at+1 reward rt−1 reward rt Energy of nonlocal moves… large inter-basin distancesmall basin size approximation ra...

  3. [4]

    non-backbone spins MCMC Tlow

  4. [5]

    RLNMC (to- tal)

    all spins MCMC Tlow Configuration space […,σb=1,…] […,σb=−1,…] sequence of NMC jumps GNN recurrent policy Save states , actions , and rewards at each step for GNN training with RL st at rt NMC step st−1 st st+1 backbone at−1 backbone at backbone at+1 reward rt−1 reward rt Energy of nonlocal moves… large inter-basin distancesmall basin size approximation ra...

  5. [6]

    freezing

    Time-to-solution Fig. 4 shows the TTS 99 curve of SA/NMC/RLNMC for the 80 percentile of the scale-free instances (see App. A 1 b for instance description), measured in MC sweeps, as a function of the individual replica runtime. NMC/RLNMC algorithms begin at βNMC = 5 ∈ (βi = 2, βf = 8) and follow the schedule of SA until βf . This figure illustrates the sl...

  6. [7]

    4 we also show the the average energy (number of unsatisfied clauses) for the scale free problems in Fig

    Residual energy To complement the results in Fig. 4 we also show the the average energy (number of unsatisfied clauses) for the scale free problems in Fig. 5a as a function of MC sweeps on the log-log scale. For SA, there are seemingly two phases with distinct slopes of the E vs MC sweeps curve: an initial steep stage, and the second “frozen” stage. We ob...

    2000

  7. [8]

    Diversity of solutions Figs. 4 and 5a have shown that we are able to re- duce time-to-solution and average energy across replicas metrics when improving SA with nonlocal moves (NMC) and reinforcement learning (RLNMC). In principle, this can be achieved with either (a) reliably getting the same states within the accepted approximation ratio across in- depe...

  8. [9]

    distance to best σ

    RLNMC policy features We would like to gain insights into the features of the trained RLNMC policies of this paper. Fig. 8 shows ex- amples of the energy landscape trajectory for the uniform random N = 500 (scale-free random N = 250) prob- lems. The basin energy is defined as the minimum en- ergy within every 600 (300) MC sweeps. The Hamming “distance to ...

Show all 118 references
  1. [10]

    unstable

    Simulated Annealing (MCMC SA) We limit the total number of MC sweeps Nsw that SA can run for and optimize the initial βi and final βf tem- peratures to get close to optimal (within the error bars) performance of the median TTS99 across the 64 instances used for hyperparameter ...

    2000

  2. [11]

    back- bones

    Nonlocal Monte Carlo Simulated Annealing (NMC) The Nonlocal Nonequilibrium Monte Carlo (NMC) moves suggested in [18] utilize correlations ˜Jij... ≡ atanh|⟨sisj . . .⟩|/β and/or magnetizations ˜hi ≡ atanh|⟨si⟩|/β of variables to construct the “back- bones” of basins of attracti...

    2000

  3. [12]

    VI A 1 and Sec

    Reinforcement Learning Nonlocal Monte Carlo (RLNMC) RLNMC is built on top of the MCMC SA/NMC al- gorithms of Sec. VI A 1 and Sec. VI A 2. RLNMC sub- stitutes the thresholding heuristic of NMC with a deep policy trained with RL. We use the same Nsw, Ncycles hy- perparameters co...

  4. [13]

    Mezard and A

    M. Mezard and A. Montanari, Information, Physics, and Computation (Oxford University Press, Inc., USA, 2009)

    2009

  5. [14]

    Zdeborov´ a and F

    L. Zdeborov´ a and F. Krzakala, Statistical physics of in- ference: thresholds and algorithms, Advances in Physics 65, 453 (2016)

    2016

  6. [15]

    Gamarnik, Turing in the shadows of nobel and abel: an algorithmic story behind two recent prizes (2025), arXiv:2501.15312 [math.PR]

    D. Gamarnik, Turing in the shadows of nobel and abel: an algorithmic story behind two recent prizes (2025), arXiv:2501.15312 [math.PR]

    2025 arXiv

  7. [16]

    Kirkpatrick, C

    S. Kirkpatrick, C. D. Gelatt, and M. P. Vecchi, Op- timization by simulated annealing, Science 220, 671 (1983)

    1983

  8. [17]

    D. J. Earl and M. W. Deem, Parallel tempering: The- ory, applications, and new perspectives, Phys. Chem. Chem. Phys. 7, 3910 (2005)

    2005

  9. [18]

    Mohseni, P

    N. Mohseni, P. L. McMahon, and T. Byrnes, Ising ma- chines as hardware solvers of combinatorial optimization problems, Nature Reviews Physics 4, 363 (2022)

    2022

  10. [19]

    Hauke, H

    P. Hauke, H. G. Katzgraber, W. Lechner, H. Nishimori, and W. D. Oliver, Perspectives of quantum annealing: methods and implementations, Reports on Progress in Physics 83, 054401 (2020)

    2020

  11. [20]

    F. Cai, S. Kumar, T. Van Vaerenbergh, X. Sheng, R. Liu, C. Li, Z. Liu, M. Foltin, S. Yu, Q. Xia, J. J. Yang, R. Beausoleil, W. D. Lu, and J. P. Strachan, Power-efficient combinatorial optimization using intrin- sic noise in memristor hopfield neural networks, Nature Electronic...

    2020

  12. [21]

    Albash and D

    T. Albash and D. A. Lidar, Demonstration of a scal- ing advantage for a quantum annealer over simulated annealing, Phys. Rev. X 8, 031016 (2018)

    2018

  13. [22]

    D. Gamarnik, The overlap gap property: A topological barrier to optimizing over ran- dom structures, Proceedings of the National Academy of Sciences 118, e2108492118 (2021), https://www.pnas.org/doi/pdf/10.1073/pnas.2108492118

    2021 doi

  14. [23]

    R. H. Swendsen and J.-S. Wang, Nonuniversal critical dynamics in monte carlo simulations, Phys. Rev. Lett. 58, 86 (1987)

    1987

  15. [24]

    Wolff, Collective monte carlo updating for spin sys- tems, Phys

    U. Wolff, Collective monte carlo updating for spin sys- tems, Phys. Rev. Lett. 62, 361 (1989)

    1989

  16. [25]

    Houdayer, A cluster monte carlo algorithm for 2- dimensional spin glasses, The European Physical Jour- nal B - Condensed Matter and Complex Systems 22, 479 (2001)

    J. Houdayer, A cluster monte carlo algorithm for 2- dimensional spin glasses, The European Physical Jour- nal B - Condensed Matter and Complex Systems 22, 479 (2001)

    2001

  17. [26]

    Hamze and N

    F. Hamze and N. de Freitas, From fields to trees, in Proceedings of the 20th Conference on Uncertainty in Artificial Intelligence, UAI ’04 (AUAI Press, Arlington, Virginia, USA, 2004) pp. 243–250

    2004

  18. [27]

    Selby, Efficient subgraph-based sampling of ising- type models with frustration (2014), arXiv:1409.3934 [cond-mat.stat-mech]

    A. Selby, Efficient subgraph-based sampling of ising- type models with frustration (2014), arXiv:1409.3934 [cond-mat.stat-mech]

    2014 arXiv

  19. [28]

    Z. Zhu, A. J. Ochoa, and H. G. Katzgraber, Efficient cluster algorithm for spin glasses in any space dimen- 12 Per-variable input: xt i=[σi,|Hi|] Global input: xt=[ebestsofar,T)] MLPGRU Per-variable memory GRU Global memory ht i ht+1 i ht global ht+1 global }global pooling Bac...

    2015

  20. [29]

    Hen, Solving spin glasses with optimized trees of clus- tered spins, Phys

    I. Hen, Solving spin glasses with optimized trees of clus- tered spins, Phys. Rev. E 96, 022105 (2017)

    2017

  21. [30]

    Mohseni, D

    M. Mohseni, D. Eppens, J. Strumpfer, R. Marino, V. Denchev, A. K. Ho, S. V. Isakov, S. Boixo, F. Ricci- Tersenghi, and H. Neven, Nonequilibrium monte carlo for unfreezing variables in hard combinatorial optimiza- tion (2021), arXiv:2111.13628 [cond-mat.dis-nn]

    2021 arXiv

  22. [31]

    Mohseni, M

    M. Mohseni, M. M. Rams, S. V. Isakov, D. Eppens, S. Pielawa, J. Strumpfer, S. Boixo, and H. Neven, Sampling diverse near-optimal solutions via algorithmic quantum annealing, Phys. Rev. E 108, 065303 (2023)

    2023

  23. [32]

    Bengio, A

    Y. Bengio, A. Lodi, and A. Prouvost, Machine learning for combinatorial optimization: a methodological tour d’horizon (2020), arXiv:1811.06128 [cs.LG]

    2020 arXiv

  24. [33]

    R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction (A Bradford Book, Cambridge, MA, USA, 2018)

    2018

  25. [34]

    Reinforcement learning nonlocal monte carlo, github.com/dumdob/rlnmc.git (2025)

    2025

  26. [35]

    Bello, H

    I. Bello, H. Pham, Q. V. Le, M. Norouzi, and S. Bengio, Neural combinatorial optimization with reinforcement learning (2017), arXiv:1611.09940 [cs.AI]

    2017 arXiv

  27. [36]

    Z. Li, Q. Chen, and V. Koltun, Combinatorial optimiza- tion with graph convolutional networks and guided tree search (2018), arXiv:1810.10659 [cs.LG]

    2018 arXiv

  28. [37]

    H. He, H. Daume III, and J. M. Eisner, Learning to search in branch and bound algorithms, in Advances in Neural Information Processing Systems , Vol. 27, edited by Z. Ghahramani, M. Welling, C. Cortes, N. Lawrence, and K. Weinberger (Curran Associates, Inc., 2014)

    2014

  29. [38]

    W. Kool, H. van Hoof, and M. Welling, Attention, learn to solve routing problems! (2019), arXiv:1803.08475 [stat.ML]

    2019 arXiv

  30. [39]

    H. Dai, E. B. Khalil, Y. Zhang, B. Dilkina, and L. Song, Learning combinatorial optimization algorithms over graphs (2018), arXiv:1704.01665 [cs.LG]

    2018 arXiv

  31. [40]

    T. D. Barrett, W. R. Clements, J. N. Foerster, and A. I. Lvovsky, Exploratory combinatorial optimization with reinforcement learning (2020), arXiv:1909.04063 [cs.LG]

    2020 arXiv

  32. [41]

    T. D. Barrett, C. W. F. Parsonson, and A. Lat- erre, Learning to solve combinatorial graph par- titioning problems via efficient exploration (2022), arXiv:2205.14105 [cs.LG]

    2022 arXiv

  33. [42]

    T¨ onshoff, B

    J. T¨ onshoff, B. Kisin, J. Lindner, and M. Grohe, One model, any csp: Graph neural networks as fast global search heuristics for constraint satisfaction, in Proceed- ings of the Thirty-Second International Joint Confer- ence on Artificial Intelligence, IJCAI-23 , edited by E....

    2023

  34. [43]

    Mazyavkina, S

    N. Mazyavkina, S. Sviridov, S. Ivanov, and E. Bur- naev, Reinforcement learning for combinatorial opti- mization: A survey, Computers & Operations Research 134, 105400 (2021)

    2021

  35. [44]

    S. Ahn, Y. Seo, and J. Shin, Learning what to defer for maximum independent sets (2020), arXiv:2006.09607 [cs.LG]

    2020 arXiv

  36. [45]

    Mills, P

    K. Mills, P. Ronagh, and I. Tamblyn, Finding the ground state of spin hamiltonians with reinforcement 13 learning, Nature Machine Intelligence 2, 509 (2020)

    2020

  37. [46]

    W. Kool, H. van Hoof, J. Gromicho, and M. Welling, Deep policy dynamic programming for vehicle routing problems (2021), arXiv:2102.11756 [cs.LG]

    2021 arXiv

  38. [47]

    B¨ other, O

    M. B¨ other, O. Kißig, M. Taraz, S. Cohen, K. Sei- del, and T. Friedrich, What’s wrong with deep learn- ing in tree search for combinatorial optimization (2022), arXiv:2201.10494 [cs.LG]

    2022 arXiv

  39. [48]

    J. Zhou, G. Cui, S. Hu, Z. Zhang, C. Yang, Z. Liu, L. Wang, C. Li, and M. Sun, Graph neural networks: A review of methods and applications, AI Open 1, 57 (2020)

    2020

  40. [49]

    Cappart, D

    Q. Cappart, D. Ch´ etelat, E. Khalil, A. Lodi, C. Morris, and P. Veliˇ ckovi´ c, Combinatorial optimization and rea- soning with graph neural networks, Journal of Machine Learning Research 24, 1 (2023)

    2023

  41. [50]

    Perdomo-Ortiz, A

    A. Perdomo-Ortiz, A. Feldman, A. Ozaeta, S. V. Isakov, Z. Zhu, B. O’Gorman, H. G. Katzgraber, A. Diedrich, H. Neven, J. de Kleer, B. Lackey, and R. Biswas, Readi- ness of quantum optimization machines for industrial applications, Phys. Rev. Appl. 12, 014004 (2019)

    2019

  42. [51]

    Valiante, M

    E. Valiante, M. Hernandez, A. Barzegar, and H. G. Katzgraber, Computational overhead of locality reduc- tion in binary optimization problems, Computer Physics Communications 269, 108102 (2021)

    2021

  43. [52]

    Dobrynin, A

    D. Dobrynin, A. Renaudineau, M. Hizzani, D. Strukov, M. Mohseni, and J. P. Strachan, Energy landscapes of combinatorial optimization in ising machines, Phys. Rev. E 110, 045308 (2024)

    2024

  44. [53]

    Dobrynin, M

    D. Dobrynin, M. Tedeschi, A. Heittmann, and J. P. Strachan, Gradient matching of higher order combina- torial optimization in quadratic ising machines, in 2024 IEEE International Conference on Rebooting Comput- ing (ICRC) (2024) pp. 1–10

    2024

  45. [54]

    C. Fan, M. Shen, Z. Nussinov, Z. Liu, Y. Sun, and Y.- Y. Liu, Searching for spin glass ground states through deep reinforcement learning, Nature Communications 14, 725 (2023)

    2023

  46. [55]

    M. J. A. Schuetz, J. K. Brubaker, and H. G. Katzgraber, Combinatorial optimization with physics-inspired graph neural networks, Nature Machine Intelligence 4, 367 (2022)

    2022

  47. [56]

    M. J. A. Schuetz, J. K. Brubaker, Z. Zhu, and H. G. Katzgraber, Graph coloring with physics-inspired graph neural networks, Phys. Rev. Res. 4, 043131 (2022)

    2022

  48. [57]

    Heydaribeni, X

    N. Heydaribeni, X. Zhan, R. Zhang, T. Eliassi-Rad, and F. Koushanfar, Distributed constrained combina- torial optimization leveraging hypergraph neural net- works, Nature Machine Intelligence 6, 664 (2024)

    2024

  49. [58]

    Pugacheva, A

    D. Pugacheva, A. Ermakov, I. Lyskov, I. Makarov, and Y. Zotov, Enhancing gnns performance on combinato- rial optimization by recurrent feature update (2024), arXiv:2407.16468 [cs.LG]

    2024 arXiv

  50. [59]

    Langedal and F

    K. Langedal and F. Manne, Graph neural networks as ordering heuristics for parallel graph coloring (2024), arXiv:2408.05054 [cs.LG]

    2024 arXiv

  51. [60]

    C. Hu, Assessing and enhancing graph neural net- works for combinatorial optimization: Novel approaches and application in maximum independent set problems (2024), arXiv:2411.05834 [math.OC]

    2024 arXiv

  52. [61]

    M. C. Angelini and F. Ricci-Tersenghi, Modern graph neural networks do worse than classical greedy algo- rithms in solving combinatorial optimization problems like maximum independent set, Nature Machine Intelli- gence 5, 29 (2023)

    2023

  53. [62]

    S. Boettcher, Inability of a graph neural network heuris- tic to outperform greedy algorithms in solving combi- natorial optimization problems, Nature Machine Intel- ligence 5, 24 (2023)

    2023

  54. [63]

    Boettcher, Deep reinforced learning heuristic tested on spin-glass ground states: The larger picture, Nature Communications 14, 5658 (2023)

    S. Boettcher, Deep reinforced learning heuristic tested on spin-glass ground states: The larger picture, Nature Communications 14, 5658 (2023)

    2023

  55. [64]

    D. Gamarnik, Barriers for the performance of graph neural networks (gnn) in discrete ran- dom structures, Proceedings of the National Academy of Sciences 120, e2314092120 (2023), https://www.pnas.org/doi/pdf/10.1073/pnas.2314092120

    2023 doi

  56. [65]

    D. Wu, L. Wang, and P. Zhang, Solving statistical me- chanics using variational autoregressive networks, Phys. Rev. Lett. 122, 080602 (2019)

    2019

  57. [66]

    K. A. Nicoli, S. Nakajima, N. Strodthoff, W. Samek, K.-R. M¨ uller, and P. Kessel, Asymptotically unbiased estimation of physical observables with neural samplers, Phys. Rev. E 101, 023304 (2020)

    2020

  58. [67]

    McNaughton, M

    B. McNaughton, M. V. Miloˇ sevi´ c, A. Perali, and S. Pi- lati, Boosting monte carlo simulations of spin glasses us- ing autoregressive neural networks, Phys. Rev. E 101, 053312 (2020)

    2020

  59. [68]

    D. Wu, R. Rossi, and G. Carleo, Unbiased monte carlo cluster updates with autoregressive neural networks, Phys. Rev. Res. 3, L042024 (2021)

    2021

  60. [69]

    Hibat-Allah, E

    M. Hibat-Allah, E. M. Inack, R. Wiersema, R. G. Melko, and J. Carrasquilla, Variational neural annealing, Na- ture Machine Intelligence 3, 952 (2021)

    2021

  61. [70]

    Ahsan Khandoker, J

    S. Ahsan Khandoker, J. Munshad Abedin, and M. Hibat-Allah, Supplementing recurrent neural net- works with annealing to solve combinatorial optimiza- tion problems, Machine Learning: Science and Technol- ogy 4, 015026 (2023)

    2023

  62. [71]

    Sanokowski, W

    S. Sanokowski, W. Berghammer, S. Hochreiter, and S. Lehner, Variational annealing on graphs for com- binatorial optimization, in Advances in Neural Infor- mation Processing Systems , Vol. 36, edited by A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine (Curran...

    2023

  63. [72]

    Q. Ma, Z. Ma, J. Xu, H. Zhang, and M. Gao, Mes- sage passing variational autoregressive network for solv- ing intractable ising models, Communications Physics7, 236 (2024)

    2024

  64. [73]

    F. Pan, P. Zhou, H.-J. Zhou, and P. Zhang, Solving statistical mechanics on sparse graphs with feedback-set variational autoregressive networks, Phys. Rev. E 103, 012103 (2021)

    2021

  65. [74]

    Biazzo, The autoregressive neural network architec- ture of the boltzmann distribution of pairwise inter- acting spins systems, Communications Physics 6, 296 (2023)

    I. Biazzo, The autoregressive neural network architec- ture of the boltzmann distribution of pairwise inter- acting spins systems, Communications Physics 6, 296 (2023)

    2023

  66. [75]

    Biazzo, D

    I. Biazzo, D. Wu, and G. Carleo, Sparse autoregres- sive neural networks for classical spin systems, Machine Learning: Science and Technology 5, 025074 (2024)

    2024

  67. [76]

    L. M. Del Bono, F. Ricci-Tersenghi, and F. Zamponi, Nearest-neighbors neural network architecture for effi- cient sampling of statistical physics models, Machine Learning: Science and Technology 6, 025029 (2025)

    2025

  68. [77]

    Ciarella, J

    S. Ciarella, J. Trinquier, M. Weigt, and F. Zamponi, Machine-learning-assisted monte carlo fails at sampling computationally hard problems, Machine Learning: Sci- 14 ence and Technology 4, 010501 (2023)

    2023

  69. [78]

    D. Ghio, Y. Dandi, F. Krzakala, and L. Zde- borov´ a, Sampling with flows, diffusion, and autoregressive neural networks from a spin- glass perspective, Proceedings of the National Academy of Sciences 121, e2311810121 (2024), https://www.pnas.org/doi/pdf/10.1073/pnas.2311810121

    2024 doi

  70. [79]

    L. M. D. Bono, F. Ricci-Tersenghi, and F. Zamponi, On the performance of machine-learning-assisted monte carlo in sampling from simple statistical physics models (2025), arXiv:2505.22598 [cond-mat.dis-nn]

    2025

  71. [80]

    M. R. Garey and D. S. Johnson, Computers and In- tractability; A Guide to the Theory of NP-Completeness (W. H. Freeman & Co., USA, 1990)

    1990

  72. [81]

    Bybee, D

    C. Bybee, D. Kleyko, D. E. Nikonov, A. Khosrowshahi, B. A. Olshausen, and F. T. Sommer, Efficient optimiza- tion with higher-order ising machines, Nature Commu- nications 14, 6033 (2023)

    2023

  73. [82]

    Sharma, M

    A. Sharma, M. Burns, A. Hahn, and M. Huang, Augmenting an electronic ising machine to effectively solve boolean satisfiability, Scientific Reports 13, 22858 (2023)

    2023

  74. [83]

    Bhattacharya, G

    T. Bhattacharya, G. H. Hutchinson, G. Pedretti, X. Sheng, J. Ignowski, T. Van Vaerenbergh, R. Beau- soleil, J. P. Strachan, and D. B. Strukov, Computing high-degree polynomial gradients in memory, Nature Communications 15, 8211 (2024)

    2024

  75. [84]

    Nikhar, S

    S. Nikhar, S. Kannan, N. A. Aadit, S. Chowdhury, and K. Y. Camsari, All-to-all reconfigurability with sparse and higher-order ising machines, Nature Communica- tions 15, 8977 (2024)

    2024

  76. [85]

    Pedretti, F

    G. Pedretti, F. B¨ ohm, T. Bhattacharya, A. Heittmann, X. Zhang, M. Hizzani, G. Hutchinson, D. Kwon, J. Moon, E. Valiante, I. Rozada, C. E. Graves, J. Ig- nowski, M. Mohseni, J. P. Strachan, D. Strukov, R. Beausoleil, and T. Van Vaerenbergh, Solving boolean satisfiability prob...

    2025

  77. [86]

    W.-K. Chen, D. Gamarnik, D. Panchenko, and M. Rah- man, Suboptimality of local algorithms for a class of max-cut problems, The Annals of Probability 47, 1587 (2019)

    2019

  78. [87]

    Marino, G

    R. Marino, G. Parisi, and F. Ricci-Tersenghi, The back- tracking survey propagation algorithm for solving ran- dom k-sat problems, Nature Communications 7, 12996 (2016)

    2016

  79. [88]

    Gamarnik, C

    D. Gamarnik, C. Moore, and L. Zdeborov´ a, Disordered systems insights on computational hardness, Journal of Statistical Mechanics: Theory and Experiment 2022, 114015 (2022)

    2022

  80. [89]

    Karimi, G

    H. Karimi, G. Rosenberg, and H. G. Katzgraber, Ef- fective optimization using sample persistence: A case study on quantum annealers and various monte carlo optimization methods, Phys. Rev. E 96, 043312 (2017)

    2017

  81. [90]

    Bravyi, A

    S. Bravyi, A. Kliesch, R. Koenig, and E. Tang, Obsta- cles to variational quantum optimization from symme- try protection, Phys. Rev. Lett. 125, 260505 (2020)

    2020

  82. [91]

    Bravyi, A

    S. Bravyi, A. Kliesch, R. Koenig, and E. Tang, Hybrid quantum-classical algorithms for approximate graph coloring, Quantum 6, 678 (2022)

    2022

  83. [92]

    J. R. Finˇ zgar, A. Kerschbaumer, M. J. Schuetz, C. B. Mendl, and H. G. Katzgraber, Quantum-informed re- cursive optimization algorithms, PRX Quantum 5, 020327 (2024)

    2024

  84. [93]

    Bradbury, R

    J. Bradbury, R. Frostig, P. Hawkins, M. J. Johnson, C. Leary, D. Maclaurin, G. Necula, A. Paszke, J. Van- derPlas, S. Wanderman-Milne, and Q. Zhang, JAX: composable transformations of Python+NumPy pro- grams (2018)

    2018

  85. [94]

    M. C. Angelini, M. Avila-Gonz´ alez, F. D’Amico, D. Machado, R. Mulet, and F. Ricci-Tersenghi, Algo- rithmic thresholds in combinatorial optimization de- pend on the time scaling (2025), arXiv:2504.11174 [cond-mat.dis-nn]

    2025

  86. [95]

    Biere, A

    A. Biere, A. Biere, M. Heule, H. van Maaren, and T. Walsh, Handbook of Satisfiability: Volume 185 Fron- tiers in Artificial Intelligence and Applications (IOS Press, NLD, 2009)

    2009

  87. [96]

    Huembeli, J

    P. Huembeli, J. M. Arrazola, N. Killoran, M. Mohseni, and P. Wittek, The physics of energy-based models, Quantum Machine Intelligence 4, 1 (2022)

    2022

  88. [97]

    Niazi, S

    S. Niazi, S. Chowdhury, N. A. Aadit, M. Mohseni, Y. Qin, and K. Y. Camsari, Training deep boltzmann networks with sparse ising machines, Nature Electronics 7, 610 (2024)

    2024

  89. [98]

    Chowdhury, A

    S. Chowdhury, A. Grimaldi, N. A. Aadit, S. Niazi, M. Mohseni, S. Kanai, H. Ohno, S. Fukami, L. Theog- arajan, G. Finocchio, S. Datta, and K. Y. Camsari, A full-stack view of probabilistic computing with p-bits: Devices, architectures, and algorithms, IEEE Journal on Explorator...

    2023

  90. [99]

    N. A. Aadit, M. Mohseni, and K. Y. Camsari, Accelerat- ing adaptive parallel tempering with fpga-based p-bits, in 2023 IEEE Symposium on VLSI Technology and Cir- cuits (VLSI Technology and Circuits) (2023) pp. 1–2

    2023

  91. [100]

    Chowdhury, N

    S. Chowdhury, N. A. Aadit, A. Grimaldi, E. Raimondo, A. Raut, P. A. Lott, J. H. Mentink, M. M. Rams, F. Ricci-Tersenghi, M. Chiappini, L. S. Theogarajan, T. Srimani, G. Finocchio, M. Mohseni, and K. Y. Cam- sari, Pushing the boundary of quantum advantage in hard combinatorial ...

    2025

  92. [101]

    Mohseni, A

    M. Mohseni, A. Scherer, K. G. Johnson, O. Wertheim, M. Otten, N. A. Aadit, Y. Alexeev, K. M. Bresniker, K. Y. Camsari, B. Chapman, S. Chatterjee, G. A. Dag- new, A. Esposito, F. Fahim, M. Fiorentino, A. Gaj- jar, A. Khalid, X. Kong, B. Kulchytskyy, E. Kyoseva, R. Li, P. A. Lot...

    2025 arXiv

  93. [102]

    Vaswani, N

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin, At- tention is all you need (2023), arXiv:1706.03762 [cs.CL]

    2023 arXiv

  94. [103]

    Mohseni, N

    M. Mohseni, N. A. Aadit, and A. Lott, Nonlo- cal monte carlo, github.com/usra-riacs/nonlocal-monte- carlo (2023)

    2023

  95. [104]

    Montanari, F

    A. Montanari, F. Ricci-Tersenghi, and G. Semerjian, Clusters of solutions and replica symmetry breaking in random k-satisfiability, Journal of Statistical Mechanics: Theory and Experiment 2008, P04004 (2008)

    2008

  96. [105]

    Ans´ otegui, M

    C. Ans´ otegui, M. L. Bonet, and J. Levy, Towards industrial-like random sat instances, in Proceedings of 15 the 21st International Joint Conference on Artificial Intelligence, IJCAI’09 (Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 2009) pp. 387–392

    2009

  97. [106]

    Friedrich, A

    T. Friedrich, A. Krohmer, R. Rothenberger, and A. M. Sutton, Phase transitions for scale-free sat formulas, in Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, AAAI’17 (AAAI Press, 2017) pp. 3893–3899

    2017

  98. [107]

    Ignatiev, A

    A. Ignatiev, A. Morgado, and J. Marques-Silva, PySAT: A Python toolkit for prototyping with SAT oracles, in SAT (2018) pp. 428–437

    2018

  99. [108]

    T. F. Rønnow, Z. Wang, J. Job, S. Boixo, S. V. Isakov, D. Wecker, J. M. Martinis, D. A. Lidar, and M. Troyer, Defining and detecting quantum speedup, Science 345, 420 (2014)

    2014

  100. [109]

    Gurobi Optimization, LLC, Gurobi Optimizer Refer- ence Manual (2025)

    2025

  101. [110]

    Schulman, S

    J. Schulman, S. Levine, P. Moritz, M. I. Jordan, and P. Abbeel, Trust region policy optimization (2017), arXiv:1502.05477 [cs.LG]

    2017 arXiv

  102. [111]

    Schulman, F

    J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, Proximal policy optimization algorithms (2017), arXiv:1707.06347 [cs.LG]

    2017 arXiv

  103. [112]

    C. Lu, J. Kuba, A. Letcher, L. Metz, C. Schroeder de Witt, and J. Foerster, Discovered policy optimisation, Advances in Neural Information Processing Systems35, 16455 (2022)

    2022

  104. [113]

    J. Heek, A. Levskaya, A. Oliver, M. Ritter, B. Ron- depierre, A. Steiner, and M. van Zee, Flax: A neural network library and ecosystem for JAX (2024)

    2024

  105. [114]

    R. T. Lange, gymnax: A JAX-based reinforcement learning environment library (2022)

    2022

  106. [115]

    Babuschkin, K

    DeepMind, I. Babuschkin, K. Baumli, A. Bell, S. Bhu- patiraju, J. Bruce, P. Buchlovsky, D. Budden, T. Cai, A. Clark, I. Danihelka, A. Dedieu, C. Fantacci, J. God- win, C. Jones, R. Hemsley, T. Hennigan, M. Hessel, S. Hou, S. Kapturowski, T. Keck, I. Kemaev, M. King, M. Kunesch...

    2020

  107. [116]

    G. Zhou, A. Dedieu, N. Kumar, W. Lehrach, M. L´ azaro- Gredilla, S. Kushagra, and D. George, Pgmax: Factor graphs for discrete probabilistic graphical models and loopy belief propagation in jax (2023), arXiv:2202.04110 [cs.LG]. ACKNOWLEDGMENTS This material is based upon work ...

    2023 arXiv

  108. [117]

    unifor- mity

    4-SA T benchmarks a. Uniform random 4-SAT in the rigidity phase Uniform random k-SAT problems are a common com- binatorial optimization benchmark exhibiting a rich va- riety of phase transition phenomena [1]. The “unifor- mity” of this class (as opposed to the scale-free probl...

    2000

  109. [118]

    Time-to-solution One quantity of interest in this paper is time-to- solution Eq

    Metrics a. Time-to-solution One quantity of interest in this paper is time-to- solution Eq. 6, which consists of a product of a mono- tonically increasing term τ (Nsw) and a monotonically decreasing term log (1 − 0.99)/ log (1 − p(Nsw)). The re- sulting typically observed curv...

  110. [119]

    horizontal

    Reinforcement learning details Proximal policy optimization (PPO) is a reinforcement learning algorithm within the large family of policy gra- dient methods. PPO clips an RL objective function so that during training updates a new policy πθ is not too far from the old πθold , ...

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.