REVIEW 4 major objections 6 minor 51 references
A reinforcement-learning agent finds transition-state geometries by walking the energy surface from reactant to product, without labelled paths or expert guesses.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
Soft actor-critic RL locates first-order saddle points on realistic molecular PESs from reactant and product alone, without labelled paths or hand-crafted reaction coordinates.
T0 review reviewed 2026-07-14 challenge →
load-bearing objection Solid SAC agent for TS search on small GFN2-xTB systems; the “pathway-free” framing is a bit oversold because the product vector steers both actions and reward, but the engineering and benchmarks are real. the 4 major comments →
Autonomous Transition State Search with Soft Actor-Critic Reinforcement Learning
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
Core claim
Without labelled trajectories or prescribed reaction pathways, a soft actor-critic agent that navigates a molecular potential energy surface from reactant toward product using only local gradients, Hessian eigenvalues, and a product-directed target vector can locate first-order saddle-point geometries that agree with established optimizers to chemical accuracy on standard benchmark reactions.
What carries the argument
The soft actor-critic transition-state search (SACTS) agent: a continuous-action policy that proposes internal-coordinate displacements as a learned combination of the softest Hessian mode, a gradient-steering direction, and a global step-size controller, trained by a multi-component reward that rewards progress toward the product, single-imaginary-mode regions, and low RMS gradient, with a two-phase curriculum that zooms from broad exploration into tight refinement once a candidate saddle zone is found.
Load-bearing premise
That a single product-directed target vector plus a hand-designed reward is enough to keep the agent on the chemically intended path when the molecule rearranges a lot, rather than letting the target overwhelm the curvature signals and send the agent to an unphysical or off-pathway saddle.
What would settle it
Run SACTS on a larger set of reactions that involve substantial geometric reorganization (for example multi-step rearrangements or ring openings without masking) and check whether the recovered saddles still match reference first-order saddle points to chemical accuracy, or whether they systematically land on competing pathways or unphysical geometries.
If this is right
- Transition-state searches can be started from reactant and product alone, without expert initial guesses or precomputed path interpolations.
- The same agent architecture can be reused across chemically different systems once the product geometry is supplied, reducing system-specific tuning.
- Candidate structures that fall short of strict convergence remain usable as high-quality starting points for conventional optimizers.
- Progressive rendering (coarse exploration followed by local refinement) can cut the number of expensive electronic-structure evaluations needed to locate a saddle.
- Reinforcement learning becomes a practical component of automated reaction-network exploration pipelines.
Where Pith is reading between the lines
- Transfer learning of a policy pretrained on small systems could amortize the cost of training a fresh agent for every new reaction.
- Adding explicit connectivity or bonding descriptors to the state would help the agent distinguish early versus late transition states and competing pathways.
- The same curriculum-plus-reward design could be tested on higher-level electronic-structure methods once the agent’s step budget is shown to remain modest.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript introduces SACTS, a soft actor-critic reinforcement-learning agent that locates first-order saddle points on molecular potential energy surfaces. Starting from a reactant geometry and knowing the product, the agent acts in continuous internal-coordinate space using a three-component action (soft-mode scaling, gradient-steering scaling, and a global step-size multiplier), a multi-component reward that encourages progress toward the product and a single-imaginary-mode region, and a two-phase progressive-rendering curriculum that tightens step sizes after a candidate TS zone is found. Electronic structure is evaluated on-the-fly with GFN2-xTB. On eight small benchmark reactions the method recovers geometries with RMSDs typically ≤0.04 Å relative to Berny references and outperforms linear and IDPP NEB on several cases where NEB fails or converges to the product. Code is deposited on GitHub.
Significance. If the central results hold under clearer framing, the work is a useful contribution to automated reaction-path discovery: it demonstrates that an off-policy maximum-entropy actor-critic agent can climb realistic molecular PESs using only local energy, gradient, and Hessian information, without labelled trajectories or a precomputed path of images. Strengths that should be credited include (i) direct interaction with a quantum-chemical environment rather than a pre-discretized grid or a pretrained MLIP, (ii) a continuous internal-coordinate action basis with progressive coarse-to-fine rendering, (iii) systematic comparison to Berny and NEB on the same surface, and (iv) public code. These elements establish a reproducible baseline for RL-based TS search even if transfer across systems and higher-level PESs remain future work.
major comments (4)
- Abstract and Action Representation: The claim that SACTS operates “without … prescribed reaction pathways” is overstated. The soft-mode eigenvector is sign-aligned by a positive projection onto the full reactant-to-product internal-coordinate displacement d_target, the same vector supplies the dense progress reward r_progress = ||d_target||_{t−1} − ||d_target||_t, and for ring opening the authors further mask spectator Z-matrix entries so that the prior remains chemically relevant. This is endpoint-conditioned geometric guidance of the same family used by double-ended methods. The manuscript should rephrase the abstract, introduction, and conclusions to state accurately that both endpoints are known and that a product-directed prior is built into the action basis and reward, while still distinguishing the approach from image-based path interpolation and from supervised trajectory learnin
- Action Representation and Results (HF + ethene; ring opening): Because d_target is load-bearing, the paper needs an ablation that quantifies how much performance depends on it. At minimum, report success rates, RMSDs, and episode counts with (i) the full target vector, (ii) the masked variant, and (iii) a pure soft-mode / gradient-steering policy with the progress term removed or replaced by a curvature-only objective. The HF-addition case (RMSD 0.19 Å, ΔE 7.63 kcal mol⁻¹) and the authors’ own remark that a large target vector can overwhelm curvature information make this ablation essential to support the claim of curvature-informed autonomous discovery rather than guided interpolation.
- Results and Conclusions: A new agent is trained from scratch for each reaction, yet the abstract and closing paragraphs present SACTS as a “general strategy” for “scalable, automated reaction discovery” and “transferable” search. The demonstrated scope is eight systems of ≤10 atoms on GFN2-xTB with per-system training. The claims should be scaled to what is shown, and any transfer or multi-task experiment (even a simple fine-tuning from one isomerisation to another) should be added or explicitly deferred as future work rather than implied by the present results.
- Results / Figure 4 and Computational Details: The NEB baselines use linear and IDPP interpolation without climbing-image refinement. The manuscript itself cites literature in which CI-NEB with internal-coordinate interpolation succeeds on comparable systems. Either add CI-NEB (or an equivalent strong double-ended baseline) under the same GFN2-xTB forces, or qualify the comparison so that SACTS is not presented as systematically superior to the best standard double-ended practice on these surfaces.
minor comments (6)
- Introduction: Several recent RL-for-PES and generative-TS papers are cited; a short explicit contrast table (grid Q-learning, actor-critic NEB, transfer-learning agents, SACTS) would help readers place the contribution.
- State Representation, Eq. (2)–(4): The repulsive proximity signal and the tanh-filtered eigenvalues are described only briefly; a one-sentence statement of units and typical numerical ranges would aid reproducibility.
- Reward Function: The main text defers the full formula to the SI. A compact equation or table of the additive terms and their phase-dependent coefficients in the main text would make the curriculum easier to follow without leaving the paper.
- Figure 2–3: Colour scales and axis labels are hard to read in the manuscript rendering; ensure high-contrast labels and a colour-blind-safe palette in the final figures.
- Throughout: Occasional typographical issues (e.g., “user s expertise”, spacing in author names, mixed en-dashes) should be cleaned in copy-editing.
- Data Availability: The GitHub link is welcome; please also archive a frozen commit or DOI (Zenodo/Figshare) so that the exact version used for the reported numbers remains citable after future code changes.
Circularity Check
No load-bearing circularity; empirical RL search uses endpoints by design and is validated against independent Berny/NEB references.
full rationale
The paper presents an empirical soft-actor-critic agent that interacts with a GFN2-xTB PES, using reactant and product geometries to construct a target displacement vector that signs the soft mode, supplies a dense progress reward, and appears in the state. This is transparent double-ended guidance, not a derivation that reduces the reported TS geometries to the inputs by construction. Success is measured by external criteria (RMSD and energy differences versus Berny-optimized saddles, plus NEB comparisons) and by the standard first-order saddle conditions (single negative Hessian eigenvalue, gradient norm < 0.005 Ha/Bohr). No parameters are fitted to the target TS data and then re-presented as predictions; no uniqueness theorem or ansatz is imported via self-citation to force the result; the reward is hand-designed to encourage the known topological signature of a first-order saddle, which is ordinary RL goal specification rather than circular self-definition. Minor self-citations to the authors’ prior ML reviews exist but are not load-bearing for the central claim. The method is therefore self-contained against external benchmarks; the only mild tension is rhetorical (abstract language of “no prescribed reaction pathways” versus the explicit product-directed vector), which does not constitute a circular reduction of equations or results.
Axiom & Free-Parameter Ledger
free parameters (4)
- phase-dependent step-size ceilings (bonds/angles/dihedrals)
- reward coefficients (progress, perpendicular-gradient penalty, N_neg penalties, sparse TS-zone bonus, terminal rewards)
- SAC temperature target entropy and network architecture sizes
- gradient-norm and eigenvalue-count thresholds for phase transition and episode termination
axioms (4)
- domain assumption A first-order saddle point is uniquely identified by exactly one negative Hessian eigenvalue and a near-zero gradient norm.
- domain assumption GFN2-xTB supplies a sufficiently realistic PES topology for the eight benchmark reactions.
- ad hoc to paper The softest Hessian eigenvector (sign-aligned with the reactant-to-product vector) plus the perpendicular gradient component form a useful continuous action basis.
- ad hoc to paper Stacking five consecutive observation vectors supplies adequate short-term memory for the policy.
invented entities (1)
-
SACTS agent (soft-actor-critic policy + critic trained with the specific multi-component reward and progressive-rendering curriculum)
no independent evidence
Cite this review
Pith. "Pith review of Autonomous Transition State Search with Soft Actor-Critic Reinforcement Learning." pith.science (2026). https://pith.science/paper/XF2URLZC
@misc{pith2026260710854,
author = {Pith},
title = {Pith review of: Autonomous Transition State Search with Soft Actor-Critic Reinforcement Learning},
year = {2026},
howpublished = {\url{https://pith.science/paper/XF2URLZC}},
note = {Machine review of arXiv:2607.10854}
}
read the original abstract
Transition state (TS) search is a crucial step in understanding chemical reactivity and mechanisms, yet conventional algorithms remain computationally intensive and heavily reliant on initial guesses, user s expertise, and chemical intuition. While recent machine learning approaches have shown promise, they demand either large training datasets or geometric interpolation between known endpoints, limiting their generality. In this work, we introduce a TS search model based on the soft actor-critic model, an advanced reinforcement learning algorithm in which an agent learns to navigate potential energy surfaces directly from local energetic and curvature information starting from a given reactant and its corresponding product. By formulating the search as a sequential decision-making process in internal coordinates, the agent adaptively proposes chemically meaningful structural updates through a reward function designed to promote movement towards saddle point regions. Without labelled trajectories or prescribed reaction pathways, the method successfully identifies TS geometries for standard benchmark reactions, operating directly on realistic molecular potential energy surfaces. These results highlight the potential of RL as a general strategy for reducing dependence on initial guesses and enabling scalable, automated reaction discovery across diverse chemical systems.
Figures
Reference graph
Works this paper leans on
-
[1]
G. N. Simm and M. Reiher, Journal of Chemical Theory and Computation, 2017, 13, 6108-6119
2017
-
[2]
J. P . Unsleber, S. A. Grimmel and M. Reiher, Journal of Chemical Theory and Computation, 2022, 18, 5393-5409
2022
-
[3]
J. Yin, W. Li, H. Chen, J. Qiu, H. Feng, X. Xu, Q. Jin and X. Wang, ACS Catalysis, 2025, 15, 15754-15764
2025
-
[4]
S. M. Paul, D. S. Mytelka, C. T. Dunwiddie, C. C. Persinger, B. H. Munos, S. R. Lindborg and A. L. Schacht, Nature Reviews Drug Discovery, 2010, 9, 203-214
2010
-
[5]
K. D. Vogiatzis, M. V . Polynski, J. K. Kirkland, J. Townsend, A. Hashemi, C. Liu and E. A. Pidko, Chemical Reviews, 2019, 119, 2453-2523
2019
-
[6]
T. Das, S. Ghule and K. Vanka, ACS Central Science, 2019, 5, 1532-1540
2019
-
[7]
J. P . Unsleber and M. Reiher, Annual Review of Physical Chemistry, 2020, 71, 121-142
2020
-
[8]
Va z a r t , D
F. Va z a r t , D. C a l d e r i n i , C. Pu z z a r i n i , D. S ko u t e r i s a n d V. B a r o n e, J o u r n a l o f C h e m i c a l Theory and Computation, 2016, 12, 5385-5397
2016
-
[9]
J. S. Yoo, Y. Liu, X. Rong and A. M. Kolpak, The Journal of Physical Chemistry Letters, 2018, 9, 1473-1479
2018
-
[10]
Prozument, J
K. Prozument, J. H. Baraban, P . B. Changala, G. B. Park, R. G. Shaver, J. S. Muenter, S. J. Klippenstein, V . Y . Chernyak and R. W. Field, Proceedings of the National Academy of Sciences, 2020, 117, 146-151
2020
-
[11]
L i u , D
Y. L i u , D. M . S a n c h e z , M . R . Wa r e, E . G. C h a m p e n o i s , J. Ya n g , J. P. F. N u n e s , A . A t t a r, M . Centurion, J. P . Cryan, R. Forbes, K. Hegazy, M. C. Ho]mann, F . Ji, M. F . Lin, D. Luo, S. K. Saha, X. Shen, X. J. Wang, T. J. Martínez and T. J. A. Wolf, Nature Communications, 2023, 14,
2023
-
[12]
H. B. Schlegel, WIREs Computational Molecular Science, 2011, 1, 790-809
2011
-
[13]
H. B. Schlegel, Journal of Computational Chemistry, 1982, 3, 214-218
1982
-
[14]
Maeda, Y
S. Maeda, Y . Harabuchi, M. Takagi, T. Taketsugu and K. Morokuma, The Chemical Record, 2016, 16, 2232-2248
2016
-
[15]
Henkelman, B
G. Henkelman, B. P . Uberuaga and H. Jónsson, The Journal of Chemical Physics, 2000, 113, 9901-9904
2000
-
[16]
Peters, A
B. Peters, A. Heyden, A. T. Bell and A. Chakraborty, The Journal of Chemical Physics, 2004, 120, 7877-7886
2004
-
[17]
Z. D. Pozun, K. Hansen, D. Sheppard, M. Rupp, K.-R. Müller and G. Henkelman, The Journal of Chemical Physics, 2012,
2012
-
[18]
A. A. Peterson, The Journal of Chemical Physics, 2016,
2016
-
[19]
Denzel and J
A. Denzel and J. Kastner , Journal of Chemical Theory and Computation, 2018, 14, 5777-5786
2018
-
[20]
Meyer, K
R. Meyer, K. S. Schmuck and A. W. Hauser, Journal of Chemical Theory and Computation, 2019, 15, 6513-6523
2019
-
[21]
Pattanaik, J
L. Pattanaik, J. B. Ingraham, C. A. Grambow and W. H. Green, Physical Chemistry Chemical Physics, 2020, 22, 23618-23626
2020
-
[22]
Jackson, W
R. Jackson, W. Zhang and J. Pearson, Chemical Science, 2021, 12, 10022-10040
2021
-
[23]
G. M. Jones, B. A. Smith, J. K. Kirkland and K. D. Vogiatzis, Inorganic Chemistry Frontiers, 2023, 10, 1062-1075
2023
-
[24]
Q. Zhao, Y. Han, D. Zhang, J. Wang, P . Zhong, T. Cui, B. Yin, Y. Cao, H. Jia and C. Duan, Advanced Science, 2025, 12, e06240. 20
2025
-
[25]
K. D. Vogiatzis, C. Corminboeuf, A. Nova, K. Jorner, J. Kästner, M. Meuwly, P . Schwaller, V . Böttcher, M. Drosou, E. Fako, H. Hoppe, Z. Ivkovic, N. Iwanojko, D. A. Pantazis, S. P . Schmid, K. Szenes, A. Tetenoire and M. Reiher, Journal of the American Chemical Society, 2026, 148, 9143-9155
2026
-
[26]
Käser, L
S. Käser, L. I. Vazquez-Salazar, M. Meuwly and K. Töpfer, Digital Discovery, 2023, 2, 28-58
2023
-
[27]
T. Cui, Y. Han, H. Jia, C. Duan and Q. Zhao, Scientific Data, 2025, 13,
2025
-
[28]
Z. Wu, L. Zhou, P . Hou, Y . Liu, R. Wang, T. Guo and J.-C. Liu, JACS Au, 2025, 5, 6151-6161
2025
-
[29]
M. Z. Makoś, N. Verma, E. C. Larson, M. Freindorf and E. Kraka, The Journal of Chemical Physics, 2021,
2021
-
[30]
C. Duan, Y. Du, H. Jia and H. J. Kulik, Nature Computational Science, 2023, 3, 1045-1055
2023
-
[31]
S. Kim, J. Woo and W. Y . Kim, Nature Communications, 2024, 15,
2024
-
[32]
Duan, G.-H
C. Duan, G.-H. Liu, Y . Du, T. Chen, Q. Zhao, H. Jia, C. P . Gomes, E. A. Theodorou and H. J. Kulik, Nature Machine Intelligence, 2025, 7, 615-626
2025
-
[33]
Galustian, K
L. Galustian, K. Mark, J. Karwounopoulos, M. P . P . Kovar and E. Heid, Digital Discovery, 2025, 4, 3492-3501
2025
-
[34]
S. Gow, M. Niranjan, S. Kanza and J. G. Frey, Digital Discovery, 2022, 1, 551-567
2022
-
[35]
V. M n i h , K . K a v u kc u o g l u , D. S i l v e r, A . G ra v e s, I . A n t o n o g l o u , D. W i e r s t ra a n d M . A . Riedmiller, ArXiv, 2013, abs/1312.5602
Pith/arXiv arXiv 2013
-
[36]
A. W. Mills, J. J. Goings, D. Beck, C. Yang and X. Li, Journal of Chemical Information and Modeling, 2022, 62, 3169-3179
2022
-
[37]
Pal, Data Science, 2024, 7, 73-92
A. Pal, Data Science, 2024, 7, 73-92
2024
-
[38]
W. Xu, Y. Zhao, J. Chen, Z. Wan, D. Yan, X. Zhang and R. Zhang, Journal of Computational Chemistry, 2024, 45, 487-497
2024
-
[39]
Barrett and J
R. Barrett and J. Westermayr, The Journal of Physical Chemistry Letters, 2024, 15, 349-356
2024
-
[40]
K. T. Schütt, O. T. Unke and M. Gastegger, Proceedings of the 38th International Conference on Machine Learning, 2021, 139, 9377-9388
2021
-
[41]
Kikkawa, T
T. Kikkawa, T. Kawakami, S. Yamanaka and M. Okumura, Chemistr y Letters, 2025,
2025
-
[42]
Terven, AI, 2025, 6,
J. Terven, AI, 2025, 6,
2025
-
[43]
R. S. Sutton, D. Precup and S. Singh, Artificial Intelligence, 1999, 112, 181-211
1999
-
[44]
M. E. Ororbia and G. P . Warn, Journal of Computing and Information Science in Engineering, 2021,
2021
-
[45]
Z. Hou, K. Zhang, Y . Wan, D. Li, C. Fu and H. Yu, ArXiv, 2020, 2002.02829
Pith/arXiv arXiv 2020
-
[46]
Haarnoja, A
T. Haarnoja, A . Zhou, P. Abbeel and S. Levine, Proceedings of the 35th International Conference on Machine Learning, 2018, 1861-1870
2018
-
[47]
Bannwarth, E
C. Bannwarth, E. Caldeweyher, S. Ehlert, A. Hansen, P . Pracht, J. Seibert, S. Spicher and S. Grimme, WIREs Computational Molecular Science, 2021, 11, e1493
2021
-
[48]
Hjorth Larsen, J
A. Hjorth Larsen, J. Jørgen Mortensen, J. Blomqvist, I. E. Castelli, R. Christensen, M. Dułak, J. Friis, M. N. Groves, B. Hammer, C. Hargus, E. D. Hermes, P . C. Jennings, P . Bjerre Jensen, J. Kermode, J. R. Kitchin, E. Leonhard Kolsbjerg, J. Kubal, K. Kaasbjerg, S. Lysgaard, J. Bergmann Maronsson, T. Maxson, T. Olsen, L. Pastewka, A. Peterson, C. Rostga...
2017
-
[49]
Baker and F
J. Baker and F. Chan, Journal of Computational Chemistry, 1996, 17, 888-904
1996
-
[50]
A. B. Birkholz and H. B. Schlegel, Journal of Computational Chemistry, 2015, 36, 1157-1166
2015
-
[51]
Smidstrup, A
S. Smidstrup, A. Pedersen, K. Stokbro and H. Jónsson, The Journal of Chemical Physics, 2014,
2014
This paper was first reviewed by grok-4.5 on July 14, 2026.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.