Pith. sign in

REVIEW 4 major objections 5 minor 41 references

Beyond Interpolation: Extrapolative Reasoning with Reinforcement Learning and Graph Neural Networks

T0 review · 4 major / 5 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Modeling logic puzzles as graphs lets reinforcement-learned agents solve instances up to 16 times larger than anything seen in training.

desk verdict Useful graph benchmark and ablations for RL extrapolation, but the absolute extrapolation numbers rest on an underspecified test-generation procedure. read the letter →

arxiv 2502.04402 v1 pith:NR5T2SQW submitted 2025-02-06 cs.LG cs.AI

Niccolò Grillo , Andrea Toccaceli , Joël Mathys , Benjamin Estermann , Stefania Fresca , Roger Wattenhofer This is my paper · ORCID
classification cs.LGcs.AI MSC 68T0768T20
keywords graphneuralnetworksreinforcementlearningextrapolationlogicpuzzlessizegeneralizationmulti-agentRLPPOrewarddesign
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that the right representational bias, reward signal, and sequential state can let a reinforcement-learning agent extrapolate to logic puzzles far beyond its training distribution. The authors model six logic puzzles as graphs, train a multi-agent PPO policy with a graph neural network on one small size, and measure how often it solves larger instances. They report that the GNN-based agent solves puzzles up to 16 times larger than the training size, while a transformer baseline and sparse rewards largely fail. The paper's central claim is that architecture inductive bias, recurrence, and reward design control whether a model reasons beyond interpolation.

What carries the argument

The load-bearing machinery is the graph interface plus the reward formulation. Each puzzle is encoded as a graph with decision-nodes (cells that take actions), meta-nodes (row/column or face constraints), and feature vectors; because the local neighborhood pattern is identical at every size, the same GCN can process larger puzzles by message passing. The reward is defined as the maximal improvement in the number of cells matching the unique solution, $Q(G) = \sum_{i} \delta(g_i, \hat{g}_i)$, with a partial variant that counts only cells not involved in a rule violation. This dense reward is what carries training signal, and the paper's ablations show it is the difference between agents that extrapolate and agents that learn nothing.

What would settle it

Generate 50 or more larger instances with controlled difficulty (for example, by fixing the number of given clues or the number of solutions), then rerun the trained agents; if the fraction solved collapses when difficulty is held constant, the reported extrapolation is an artifact of easier large instances rather than genuine rule-based reasoning.

Watch

Extended reading notes

Core claim

The central claim is that extrapolative reasoning on logic puzzles can be achieved by a graph-based multi-agent reinforcement learning system, and that three design choices determine success. First, representing puzzle cells as decision-nodes and constraints as meta-nodes in a graph lets GNNs operate on any size with the same local structure. Second, replacing sparse terminal rewards with dense rewards that measure improvement toward the unique solution—either iterative improvement or a partial variant that ignores violated cells—is necessary for learning at all. Third, a recurrent state helps at modest extrapolation sizes while a state-less policy solves more puzzles at the largest sizes. With these components, the agent solves a meaningful fraction of instances at x4, x9, and even x16 the training size, which the authors take as evidence that it has extracted the underlying rules rather than memorized patterns.

Load-bearing premise

The extrapolation percentages rest on the assumption that the 50 test puzzles at each larger size are fair, representative samples generated by the same rule set, with comparable difficulty per size; the paper does not describe how test instances are produced or how difficulty scales.

Editorial extensions

If this is right

  • If the central claim holds, a model trained on small puzzle instances can be deployed directly on larger instances of the same rule set, avoiding the need to collect training data at every size.
  • The observed advantage of GNNs over transformers with positional encodings suggests that explicitly encoding relational structure matters more than raw capacity for size extrapolation.
  • The result that state-less policies outperform recurrent ones at the largest sizes implies that for stateless puzzles, a 3-hop receptive field plus repeated application can replace an explicit memory mechanism.
  • Dense, solution-proximity-based rewards appear to be a prerequisite for learning any generalizable policy, which informs reward design in other combinatorial reasoning tasks.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: a testable extension is to vary the density of clues or the number of candidate solutions at each size; if performance tracks instance difficulty rather than size alone, the reported percentages conflate size with difficulty.
  • Beyond the paper: the same decision-node/meta-node graph encoding is a generic template for constraint satisfaction problems, so the approach likely transfers beyond the six puzzles studied here.
  • Beyond the paper: because only 50 test puzzles are used per size, the confidence intervals could hide large variance; a larger test sample or a difficulty-controlled generator would sharpen the extrapolation measure.
  • Beyond the paper: the recurrent-versus-state-less trade-off suggests a hybrid policy that uses recurrence early in an episode and drops it later, a configuration the paper does not explore.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes a graph-based, multi-agent reinforcement learning framework for solving logic puzzles from the PUZZLES benchmark. Puzzles are represented as graphs with decision-nodes for atomic cells and meta-nodes for shared constraints, and agents are trained with PPO using either a GCN or a transformer processor, in recurrent or state-less modes, and with sparse, iterative, or partial rewards. The central claim is that this approach extrapolates to puzzle sizes far outside the training distribution, including instances up to 16 times larger, and that the architecture's inductive bias, recurrence, and reward design are the key factors controlling this extrapolation.

Significance. If the extrapolation results are reliable, this is a valuable empirical contribution: it provides a graph interface that enables size generalization for six logic puzzles in an RL setting, and it performs systematic ablations of architecture, reward, and recurrence. The paper also makes its code available and builds on a public benchmark. However, the headline quantitative claim depends on an extrapolation evaluation whose test-set generation is not described, and the model-selection procedure overlaps the reported +1 extrapolation results. The significance is therefore conditional on the evaluation being made rigorous and transparent.

major comments (4)
  1. [Extrapolation Evaluation] The central claim that agents solve puzzles at x4, x9, and x16 sizes relies on 50 test puzzles per size, but the paper never states how these test instances are generated. In particular, it does not specify the difficulty parameter of Simon Tatham's generator, whether uniqueness of solutions is re-checked, whether clue density is held constant across sizes, or how per-cell hardness scales with size. If larger instances are generated with a looser uniqueness constraint, more filled-in cells, or an easier difficulty setting, then the reported solve rates could reflect a distribution shift in per-cell difficulty rather than learned extrapolation. The monotone decrease in solve rates across sizes is also consistent with any size-dependent difficulty model. Please specify the generation protocol and provide per-size difficulty diagnostics (e.g., clue density, number of solutions), or use matched generators with controlled difficulty.
  2. [Training / Table 4] Model selection is performed on a validation set that is exactly the +1 extrapolation test set. Table 4 shows, for example, that Tents has validation and +1 both at 6x6, Mosaic has both at 5x5, and Loopy has both at 5x5. Thus the reported +1 solve rates are selected on, not independent evidence of extrapolation, and the checkpoints used for the x4/x9/x16 results are chosen using that same criterion. This affects interpretation of Figures 6-8 and should be corrected by using a separate validation split (e.g., a different random set at the training size, or a size that is not later reported as an extrapolation benchmark).
  3. [Table 1] The comparison against PUZZLES baselines is not controlled: the GNN is trained on larger puzzle sizes (Tents 5x5, Lightup 5x5, Mosaic 4x4, Loopy 4x4, Net 4x4), while the baselines are trained on smaller sizes (Tents 4x4, Lightup 3x3, Mosaic 3x3, Loopy 3x3, Net 2x2). The differing training sizes alone can explain a large part of the performance gap. Before claiming superiority over the baselines, the authors should either retrain the baselines on the same puzzle sizes or explicitly report baseline performance at the GNN's training size.
  4. [Empirical Evaluation] All central comparisons in Figures 6-8 are based on only three seeds and 50 test puzzles per size. The reported 95% bootstrap intervals appear to be computed over puzzles for a fixed selected checkpoint and do not account for seed variance or for the model-selection uncertainty induced by choosing the checkpoint on the validation set. Please report per-seed results, confidence intervals across seeds, or a per-puzzle breakdown, so the reader can judge whether the x9 and x16 solve percentages are stable and not driven by a single seed or a few easy large instances.
minor comments (5)
  1. [Training Parameters and Baseline Establishment] The parameter list uses the flag 'gcn' but describes it as representing the GENConv architecture, while the main text consistently calls the model a GCN; please clarify the exact architecture used.
  2. [Reward] The sparse reward is defined as 'R reward if the game is solved', but the value of R is never specified; please state it explicitly.
  3. [Empirical Evaluation] Figures 6-8 aggregate results across six puzzles with the interquartile mean, but no per-puzzle extrapolation tables are provided; this makes it hard to assess which puzzles drive the reported effects.
  4. [Conclusion] There is a typo in the conclusion: 'approache' should be 'approach'.
  5. [Related Work] The citation to Tönshoff et al. appears with a malformed TeX accent ('T¨onshoff'); please fix the rendering.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the extrapolation claims are empirical measurements against puzzle solutions, not derivations from fitted inputs or self-citation chains.

full rationale

This paper is an empirical study rather than a derivation, and the central claims are direct measurements of agent success on held-out puzzle instances. The reward functions in the 'Reward' section use the puzzle solution to shape training, but the reported extrapolation percentages are computed by checking whether the agent actually solves test puzzles, so the prediction is not equivalent to the training signal by construction. The self-citations to the PUZZLES benchmark (Estermann et al. 2024) and to the reward-shaping technique of Tönshoff et al. (2022) are not load-bearing: PUZZLES provides the environment and baseline numbers, while the graph interface, architectures, and evaluation protocol are new contributions whose results are measured against ground-truth puzzle solutions. Model selection on a validation set one size larger is a methodological choice that could affect the +1 numbers, but it is a tuning procedure, not a fitted parameter renamed as a prediction, and it does not reduce the larger extrapolation claims to the model's inputs. The stated limitations (synthetic puzzles, model-free RL only) are honest scope restrictions, not admissions of circularity. Overall, the paper is self-contained against external puzzle generators and solution checkers, and no equation or claim reduces to its own input.

Assumptions & free parameters 3 free parameters · 4 assumptions · 1 invented entities

This is an empirical RL benchmark paper, so the ledger contains no fitted physical constants or derived quantities. The central scientific contribution depends on hand-chosen thresholds, hyperparameters, and modeling assumptions about puzzle generation and transferability, which are listed above. The meta-node is the only genuinely new representational entity, and it is a modeling convenience rather than an empirical postulate.

free parameters (3)
  • min_unique_train_instances = 40,000
    Training size for each puzzle is chosen as the smallest size with at least 40,000 unique configurations, an arbitrary threshold that determines what counts as the training distribution.
  • PPO and network hyperparameters = lr 3e-4 or 6e-5, batch 320 or 3200, hidden 32, layers 3, gamma 0.5, ent_coef 0.004, timesteps 1M to 2.4M
    Hand-tuned sequentially; final results could depend on these choices, though they are standard and reported.
  • test set size = 50 puzzles per size
    Fixed by the authors; affects the reliability of solved-percentage estimates.
assumptions (4)
  • domain assumption The six selected puzzles satisfy all five selection criteria (atomic actions, independence, solvability, complete information, value generalization).
    The selection criteria in the Methodology section justify why these puzzles are suitable for extrapolation; if any puzzle violates them, the graph interface would not cleanly support size scaling.
  • domain assumption Larger puzzle instances obey exactly the same rules and have the same local graph structure as training instances.
    Stated in the graph modeling section; this is necessary for the GNN weights to transfer across sizes.
  • ad hoc to paper Dense reward computed from the known solution is a valid learning signal that encourages generalizable rules rather than memorized corrections.
    The reward section defines Q(G) as similarity to the ground-truth solution; whether this shapes generalizable behavior is an empirical assumption, not a proven property.
  • domain assumption A 3-layer GCN and a 3-layer transformer are representative implementations of their respective inductive biases.
    The architecture comparison treats these specific instantiations as fair representatives, which is a common but nontrivial modeling choice.
invented entities (1)
  • meta-node graph abstraction
    purpose: Encodes shared constraints and violation flags for a group of decision-nodes in the graph representation.
    An abstraction introduced by the paper's interface; it has no falsifiable prediction outside the benchmark and its usefulness is shown only within this study.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Beyond Interpolation: Extrapolative Reasoning with Reinforcement Learning and Graph Neural Networks." pith.science (2026). https://pith.science/paper/NR5T2SQW

@misc{pith2026250204402,
  author       = {Pith},
  title        = {Pith review of: Beyond Interpolation: Extrapolative Reasoning with Reinforcement Learning and Graph Neural Networks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/NR5T2SQW}},
  note         = {Machine review of arXiv:2502.04402}
}
read the original abstract

Despite incredible progress, many neural architectures fail to properly generalize beyond their training distribution. As such, learning to reason in a correct and generalizable way is one of the current fundamental challenges in machine learning. In this respect, logic puzzles provide a great testbed, as we can fully understand and control the learning environment. Thus, they allow to evaluate performance on previously unseen, larger and more difficult puzzles that follow the same underlying rules. Since traditional approaches often struggle to represent such scalable logical structures, we propose to model these puzzles using a graph-based approach. Then, we investigate the key factors enabling the proposed models to learn generalizable solutions in a reinforcement learning setting. Our study focuses on the impact of the inductive bias of the architecture, different reward systems and the role of recurrent modeling in enabling sequential reasoning. Through extensive experiments, we demonstrate how these elements contribute to successful extrapolation on increasingly complex puzzles.These insights and frameworks offer a systematic way to design learning-based systems capable of generalizable reasoning beyond interpolation.

Figures

Figures reproduced from arXiv: 2502.04402 by the authors.

Figure 1
Figure 1. We focus on logic puzzles of varying sizes in order [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Some Example puzzles of the PUZZLES library, [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. We provide a new graph interface in order to ease [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (14 more)
Figure 4
Figure 4. Figure 4: The decision-nodes, which represent the atomic cells of a game and the meta-nodes which contain shared in￾formation about a collection of nodes. Each decision-node directly influences the state of the game, as it can take an action. In Loopy, every edge of the original…
Figure 4
Figure 4. Figure 4: Illustration of the modeling of the puzzle Loopy. In this case, each decision-node (black or blue circles) corresponds [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: Illustration of testing the ability to generalize be [PITH_FULL_IMAGE:figures/full_fig_p005_5.png]
Figure 6
Figure 6. Figure 6: Percentage of puzzles solved during size extrap [PITH_FULL_IMAGE:figures/full_fig_p006_6.png]
Figure 8
Figure 8. Figure 8: A recurrent agent design brings advantages for the [PITH_FULL_IMAGE:figures/full_fig_p007_8.png]
Figure 7
Figure 7. Figure 7: Compared to the transformer model, the graph ar [PITH_FULL_IMAGE:figures/full_fig_p007_7.png]
Figure 9
Figure 9. Figure 9: Tents puzzle states: (a) New puzzle, (b) Solved [PITH_FULL_IMAGE:figures/full_fig_p009_9.png]
Figure 10
Figure 10. Figure 10: Lightup puzzle states: (a) New puzzle, (b) Solved [PITH_FULL_IMAGE:figures/full_fig_p010_10.png]
Figure 11
Figure 11. Figure 11: Mosaic puzzle states: (a) New puzzle, (b) Solved [PITH_FULL_IMAGE:figures/full_fig_p010_11.png]
Figure 12
Figure 12. Figure 12: Loopy puzzle states: (a) New puzzle, (b) Solved [PITH_FULL_IMAGE:figures/full_fig_p010_12.png]
Figure 15
Figure 15. Figure 15: Illustration of the graph and node attributes used for net. This game (together with lightup) presents the simplest [PITH_FULL_IMAGE:figures/full_fig_p011_15.png]
Figure 16
Figure 16. Figure 16: Illustration of the graph and node attributes used for loopy. In this case each decision node (black or white circles) [PITH_FULL_IMAGE:figures/full_fig_p011_16.png]
Figure 17
Figure 17. Figure 17: Illustration of the graph and node attributes used for mosaic. Here, differently from the net the nodes are cells that are [PITH_FULL_IMAGE:figures/full_fig_p011_17.png]
Figure 18
Figure 18. Figure 18: Illustration of the graph and node attributes used for tents. In this case, we have the decision nodes (blue and green [PITH_FULL_IMAGE:figures/full_fig_p012_18.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

41 extracted references · 22 canonical work pages

  1. [1]

    , " * write output.state after.block = add.period write newline

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Abbe, E.; Bengio, S.; Lotfi, A.; and Rizk, K. 2024. Generalization on the unseen, logic reasoning and degree curriculum. Journal of Machine Learning Research, 25(331): 1--58

    work page 2024

  4. [4]

    S.; Courville, A.; and Bellemare, M

    Agarwal, R.; Schwarzer, M.; Castro, P. S.; Courville, A.; and Bellemare, M. G. 2021. Deep Reinforcement Learning at the Edge of the Statistical Precipice. Advances in Neural Information Processing Systems

    work page 2021

  5. [5]

    W.; Hamrick, J

    Battaglia, P. W.; Hamrick, J. B.; Bapst, V.; Sanchez-Gonzalez, A.; Zambaldi, V.; Malinowski, M.; Tacchetti, A.; Raposo, D.; Santoro, A.; Faulkner, R.; Gulcehre, C.; Song, F.; Ballard, A.; Gilmer, J.; Dahl, G.; Vaswani, A.; Allen, K.; Nash, C.; Langston, V.; Dyer, C.; Heess, N.; Wierstra, D.; Kohli, P.; Botvinick, M.; Vinyals, O.; Li, Y.; and Pascanu, R. 2...

    arXiv 2018

  6. [6]

    G.; Naddaf , Y.; Veness , J.; and Bowling , M

    Bellemare , M. G.; Naddaf , Y.; Veness , J.; and Bowling , M. 2013. The Arcade Learning Environment: An Evaluation Platform for General Agents. Journal of Artificial Intelligence Research, 47: 253--279

    2013

  7. [7]

    Bohde, M.; Liu, M.; Saxton, A.; and Ji, S. 2024. On the Markov Property of Neural Algorithmic Reasoning: Analyses and Methods. arXiv:2403.04929

    work page Pith review arXiv 2024

  8. [8]

    Cappart, Q.; Chételat, D.; Khalil, E.; Lodi, A.; Morris, C.; and Veličković, P. 2022. Combinatorial optimization and reasoning with graph neural networks. arXiv:2102.09544

    arXiv 2022

Show all 41 references
  1. [9]

    H.; Leiserson, C

    Cormen, T. H.; Leiserson, C. E.; Rivest, R. L.; and Stein, C. 2022. Introduction to A lgorithms . The MIT Press, 4th edition

    2022

  2. [10]

    Dahl, F. A. 2001. A reinforcement learning algorithm applied to simplified two-player Texas Hold’em poker. In European Conference on Machine Learning, 85--96. Springer

    2001

  3. [11]

    B.; Zhang, Y.; Dilkina, B.; and Song, L

    Dai, H.; Khalil, E. B.; Zhang, Y.; Dilkina, B.; and Song, L. 2018. Learning Combinatorial Optimization Algorithms over Graphs. arXiv:1704.01665

    2018 arXiv

  4. [12]

    A.; Niedermayr, Y.; and Wattenhofer, R

    Estermann, B.; Lanzendörfer, L. A.; Niedermayr, Y.; and Wattenhofer, R. 2024. PUZZLES: A benchmark for neural algorithmic reasoning. arXiv preprint arXiv:2407.00401

    2024 arXiv

  5. [13]

    L.; Sutton, R

    Ghory, I.; Samuel, A. L.; Sutton, R. S.; and Tesauro, G. 2004. Reinforcement Learning in Board Games

    2004

  6. [14]

    Heinrich, J.; and Silver, D. 2016. Deep reinforcement learning from self-play in imperfect-information games. arXiv preprint arXiv:1603.01121

    2016 arXiv

  7. [15]

    Ibarz, B.; Kurin, V.; Papamakarios, G.; Nikiforou, K.; Bennani, M.; Csordás, R.; Dudzik, A.; Bošnjak, M.; Vitvitskyi, A.; Rubanova, Y.; Deac, A.; Bevilacqua, B.; Ganin, Y.; Blundell, C.; and Veličković, P. 2022. A Generalist Neural Algorithmic Learner. arXiv:2209.11142

    2022 arXiv

  8. [16]

    Jung, Y.; and Ahn, S. 2023. Triplet Edge Attention for Algorithmic Reasoning. arXiv:2312.05611

    2023 arXiv

  9. [18]

    N.; and Welling, M

    Kipf, T. N.; and Welling, M. 2017 b . Semi-Supervised Classification with Graph Convolutional Networks. arXiv:1609.02907

    2017 arXiv

  10. [19]

    Lai, M. 2015. Giraffe: Using deep reinforcement learning to play chess. arXiv preprint arXiv:1509.01549

    2015 arXiv

  11. [20]

    Mahdavi, S.; Swersky, K.; Kipf, T.; Hashemi, M.; Thrampoulidis, C.; and Liao, R. 2023. Towards Better Out-of-Distribution Generalization of Neural Algorithmic Reasoning Tasks. Transactions on Machine Learning Research

    2023

  12. [21]

    Markeeva, L.; McLeish, S.; Ibarz, B.; Bounsi, W.; Kozlova, O.; Vitvitskyi, A.; Blundell, C.; Goldstein, T.; Schwarzschild, A.; and Veličković, P. 2024. The CLRS-Text Algorithmic Reasoning Language Benchmark. arXiv:2406.04229

    2024 arXiv

  13. [22]

    Minder, J.; Grötschla, F.; Mathys, J.; and Wattenhofer, R. 2023. SALSA-CLRS: A Sparse and Scalable Benchmark for Algorithmic Reasoning. arXiv:2309.12253

    2023 arXiv

  14. [23]

    Müller, L.; Kusuma, D.; Bonet, B.; and Morris, C. 2024. Towards Principled Graph Transformers. arXiv:2401.10119

    2024 arXiv

  15. [24]

    Numeroso, D.; Bacciu, D.; and Veličković, P. 2023. Dual Algorithmic Reasoning. arXiv:2302.04496

    2023 arXiv

  16. [25]

    T.; Burch, N.; Anthony, T.; et al

    Perolat, J.; De Vylder, B.; Hennes, D.; Tarassov, E.; Strub, F.; de Boer, V.; Muller, P.; Connor, J. T.; Burch, N.; Anthony, T.; et al. 2022. Mastering the game of Stratego with model-free multiagent reinforcement learning. Science, 378(6623): 990--996

    2022

  17. [26]

    C.; Hagenbuchner, M.; and Monfardini, G

    Scarselli, F.; Gori, M.; Tsoi, A. C.; Hagenbuchner, M.; and Monfardini, G. 2008. The graph neural network model. IEEE transactions on neural networks, 20(1): 61--80

    2008

  18. [27]

    Schulman, J.; Wolski, F.; Dhariwal, P.; Radford, A.; and Klimov, O. 2017. Proximal Policy Optimization Algorithms. arXiv:1707.06347

    2017 arXiv

  19. [28]

    Schwarzschild, A.; Borgnia, E.; Gupta, A.; Huang, F.; Vishkin, U.; Goldblum, M.; and Goldstein, T. 2021. Can You Learn an Algorithm? Generalizing from Easy to Hard Problems with Recurrent Networks. arXiv:2106.04537

    2021 arXiv

  20. [29]

    Silver, D.; Huang, A.; Maddison, C. J.; Guez, A.; Sifre, L.; van den Driessche, G.; Schrittwieser, J.; Antonoglou, I.; Panneershelvam, V.; Lanctot, M.; Dieleman, S.; Grewe, D.; Nham, J.; Kalchbrenner, N.; Sutskever, I.; Lillicrap, T.; Leach, M.; Kavukcuoglu, K.; Graepel, T.; a...

    2016

  21. [30]

    Silver, D.; Hubert, T.; Schrittwieser, J.; Antonoglou, I.; Lai, M.; Guez, A.; Lanctot, M.; Sifre, L.; Kumaran, D.; Graepel, T.; et al. 2017. Mastering chess and shogi by self-play with a general reinforcement learning algorithm. arXiv preprint arXiv:1712.01815

    2017 arXiv

  22. [31]

    Silver, D.; Hubert, T.; Schrittwieser, J.; Antonoglou, I.; Lai, M.; Guez, A.; Lanctot, M.; Sifre, L.; Kumaran, D.; Graepel, T.; et al. 2018. A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play. Science, 362(6419): 1140--1144

    2018

  23. [32]

    Steinberger, E. 2019. PokerRL. https://github.com/TinkeringCode/PokerRL

    2019

  24. [33]

    Szita, I. 2012. Reinforcement learning in games. In Reinforcement Learning: State-of-the-art, 539--577. Springer

    2012

  25. [34]

    Tatham, S. 2004. Simon Tatham's Portable Puzzle Collection. Accessed: 2024-12-04

    2004

  26. [35]

    Tönshoff, J.; Kisin, B.; Lindner, J.; and Grohe, M. 2022. One Model, Any CSP: Graph Neural Networks as Fast Global Search Heuristics for Constraint Satisfaction. arXiv:2208.10227

    2022 arXiv

  27. [36]

    N.; Kaiser, L.; and Polosukhin, I

    Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A. N.; Kaiser, L.; and Polosukhin, I. 2017. Attention is all you need. Advances in Neural Information Processing Systems

    2017

  28. [37]

    P.; Budden, D.; Pascanu, R.; Banino, A.; Dashevskiy, M.; Hadsell, R.; and Blundell, C

    Velickovic, P.; Badia, A. P.; Budden, D.; Pascanu, R.; Banino, A.; Dashevskiy, M.; Hadsell, R.; and Blundell, C. 2022. The CLRS algorithmic reasoning benchmark. In Proceedings of the 39th International Conference on Machine Learning, volume 162, 22084--22102. PMLR

    2022

  29. [38]

    Veličković, P.; Cucurull, G.; Casanova, A.; Romero, A.; Liò, P.; and Bengio, Y. 2018. Graph Attention Networks. arXiv:1710.10903

    2018 arXiv

  30. [39]

    Xenou, K.; Chalkiadakis, G.; and Afantenos, S. 2019. Deep reinforcement learning in strategic board game environments. In Multi-Agent Systems: 16th European Conference, EUMAS 2018, Bergen, Norway, December 6--7, 2018, Revised Selected Papers 16, 233--248. Springer

    2019

  31. [40]

    Xu, K.; Hu, W.; Leskovec, J.; and Jegelka, S. 2019. How Powerful are Graph Neural Networks? arXiv:1810.00826

    2019 arXiv

  32. [41]

    S.; ichi Kawarabayashi, K.; and Jegelka, S

    Xu, K.; Zhang, M.; Li, J.; Du, S. S.; ichi Kawarabayashi, K.; and Jegelka, S. 2021. How Neural Networks Extrapolate: From Feedforward to Graph Neural Networks. arXiv:2009.11848

    2021 arXiv

  33. [42]

    Zhao, E.; Yan, R.; Li, J.; Li, K.; and Xing, J. 2022. AlphaHoldem: High-performance artificial intelligence for heads-up no-limit poker via end-to-end reinforcement learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, 4689--4697

    2022

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.