REVIEW 5 major objections 5 minor 11 references
Distilling the secondary-structure probabilities of a protein language model into a 45,000-parameter graph neural network yields an implicit solvent potential that, added to a standard electrostatics term, reproduces folding free-energy lan
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · deepseek-v4-flash
2026-08-03 11:39 UTC pith:N6VBJTQN
load-bearing objection A genuinely novel idea—distilling a PLM's SS8 statistics into a compact GNN for MD—with real distillation results, but the validation is softer than the abstract and the load-bearing solvation-proxy assumption is only partially tested. the 5 major comments →
Knowledge Distillation of a Protein Language Model Yields a Foundational Implicit Solvent Model
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The paper's central claim is that evolutionary information in a protein language model can stand in for water: the model's conditional probabilities of secondary-structure motifs, converted to effective energies via E = -k_BT log P, encode solvent-dominated folding preferences. A graph neural network (Schake) trained by knowledge distillation to reproduce those probabilities produces an energy term that, combined with a generalized Born electrostatics term (GBn2), reproduces explicit-solvent (TIP3P) folding free-energy profiles for several fast-folding proteins and yields extended, non-collapsed ensembles for intrinsically disordered proteins. This is presented as a transferable, foundationa
What carries the argument
The load-bearing object is the multi-state GNN energy E_ms_GNN(x) = -γ k_BT Σ_i log[max_j q_i^(j)(x)], where q_i^(j) is the GNN's predicted probability of SS8 motif j at backbone atom i and the max is smoothed by a LogSumExp to remain differentiable. Unlike a one-state energy that fixes each atom's native motif, the multi-state form lets the most probable motif switch from helix to coil as the chain unfolds, which allows one potential to describe both folded and disordered states. Schake is a multiscale graph neural network (short-range SAKE and long-range SchNet message passing over backbone Cα, C, N atoms) trained with a cross-entropy distillation loss against ESM3's SS8 predictions, and t
Load-bearing premise
The argument stands or falls on the identification of ESM3's secondary-structure probabilities with solvent-mediated folding free energies (E = -k_BT log P), an equivalence the paper asserts rather than derives; if those probabilities reflect evolutionary sequence constraints rather than solvation thermodynamics, the GNN learns a local-structure scorer, not an implicit solvent model.
What would settle it
Compute explicit-solvent (TIP3P) free-energy profiles for a held-out panel of both folded proteins and IDPs, run the hybrid model with a single fixed γ on all of them, and test whether the predicted ensembles match the references; the paper's own data (α-3D needs a larger γ; GBn2 alone matches IDPs better) already hint that per-protein rescaling may be required, which would falsify the transferability claim.
If this is right
- A single ~45,000-parameter potential can replace explicit water for both folded and disordered proteins, removing the classic ISM compromise between the two regimes.
- The distilled model runs about nine times faster than the teacher language model, so long-timescale implicit-solvent simulations become accessible on ordinary GPUs.
- Because training uses sequence–structure statistics rather than explicit-solvent trajectories, the model can be extended to new protein families without collecting new simulation data.
- In 500-ns ML/MD runs the GNN energy keeps proteins within a few Å of the native state, indicating it is stable enough to serve as a sampling engine for folded proteins.
- The multi-state energy's motif-switching behavior is the mechanism that prevents the over-compaction of disordered proteins seen in traditional GB models.
Where Pith is reading between the lines
- Editorial inference: if this works, the same distillation route should transfer to other local structural observables (backbone torsion propensities, per-residue solvent exposure, or chemical shifts), yielding GNN potentials that carry more solvation physics than SS8 alone.
- Editorial inference: the backbone-only input is the likely ceiling on accuracy; adding side-chain atoms or a separate side-chain correction would test whether missing side-chain solvation is why GBn2 alone sometimes matches IDP references better.
- Editorial inference: the need to rescale γ for alpha-3D and the slightly expanded IDP ensembles suggest the energy scale is context-dependent; a learned or coordinate-dependent γ might remove per-protein hand-tuning.
- Editorial inference: if evolutionary statistics truly encode solvent effects, this strategy implies a general recipe for building simulation potentials by distilling any high-performing structure-prediction model, bypassing the need for massive explicit-solvent training sets.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a new implicit solvent model (ISM) built by distilling secondary-structure (SS8) likelihoods from the protein language model ESM3 into a compact graph neural network (Schake, ~45k parameters). The GNN takes backbone Cα/C/N coordinates plus sequence as input and predicts per-residue SS8 probabilities. These probabilities are converted into two energy terms: a one-state energy (Eq. 1) that rewards the native SS8 pattern of a reference structure, and a multi-state energy (Eq. 2) that rewards the most probable local motif. The one-state energy is shown to correlate with RMSD on D.E. Shaw folding trajectories and to keep 500 ns MD simulations near native. The multi-state energy, when added to the GBn2 generalized Born model with a fitted scaling factor γ, is reported to reproduce TIP3P folding free-energy profiles for three fast-folding proteins and to avoid collapse for three intrinsically disordered proteins. The authors claim that this constitutes a foundational, transferable implicit solvent model applicable to both folded and disordered proteins.
Significance. If fully supported, the approach would be significant: a 45k-parameter GNN that replaces explicit-solvent thermodynamics for both folded and disordered proteins would be a major advance, and the idea of distilling ESM3 into a fast potential is creative. The paper does provide genuine evidence that a compact GNN can reproduce ESM3's SS8 predictions with only modest accuracy loss (87.0% vs 89.2%), and the correlation of the one-state energy with RMSD on D.E. Shaw trajectories is a real empirical observation. However, the physical interpretation as an implicit solvent model is not established. The teacher labels from ESM3 are sequence-only, the one-state energy is native-centric by construction, the IDP results do not show that the hybrid model outperforms GBn2 alone (the paper admits the opposite), and the scaling factor γ is fitted per regime. The central claim in the abstract is therefore only partially supported by the evidence presented.
major comments (5)
- [Eq. (1), Section “GNN-derived energies reliably distinguish folded and unfolded protein states”] The one-state energy E_os^GNN uses y_i^(j), the SS8 labels of the reference folded structure, as the target motif. Consequently the folded state is rewarded by construction; the energy is a measure of how well the current conformation matches the native SS8 pattern according to the GNN, not a solvation free energy. The correlation with RMSD shown in Fig. 3 is therefore expected: as the structure deviates from native, the GNN's probability for the native SS8 assignment will drop. This does not validate ESM3 as a solvent proxy; it validates that the GNN is a reasonable secondary-structure classifier. A control in which the native SS8 labels are shuffled or the ESM3 signal is destroyed is needed to attribute the stability to the distillation.
- [Methods, “Secondary structure targets from DISPEF and ESM3”] The paper states: “For each protein sequence, SS8 likelihoods were predicted using the ESM3-open protein language model.” Thus the teacher labels P_ESM3 are sequence-only and identical for every conformation of a given protein. The distillation loss in Eq. (7) therefore cannot impose any structure-dependent solvation behavior; the only structure-dependent training signal is the DSSP one-hot term. The resulting q_i^(j)(x) is primarily a learned structure-to-secondary-structure classifier, not an “effective energy” from ESM3. The claim in the Introduction that E = −k_BT log P(SS8|sequence) approximates the folding free-energy landscape is unsupported, because P does not depend on conformation. This is a load-bearing conceptual issue that undermines the interpretation of the model as an ISM.
- [Section “Multi-state energy formulation enables accurate simulation of folded and intrinsically disordered proteins” and] The paper acknowledges: “Notably, GBn2 without any additional correction term most-closely matches the TIP3P reference.” This directly contradicts the abstract's assertion that the hybrid model “predicts the structural ensembles of intrinsically disordered proteins.” At best, the hybrid avoids collapse; it does not reproduce the IDP ensembles quantitatively. The claim that the model “captures the secondary-structure distributions” is also weakened by the observation that the GBn2/GNN ensembles are “slightly more expanded” than TIP3P. The IDP results therefore do not support the abstract's central claim.
- [Results, γ values in Sections “GNN-Derived Energies Support Stable ML/MD Simulations” and “Multi-state energy formulatio] The scaling factor γ is set to 2.5 for standalone stability simulations, to 0.175 for the hybrid PMF calculations, and is increased further for α3D to restore a folded minimum (Fig. S9). This per-regime fitting means the model has at least one free parameter adjusted to achieve the reported agreement. The α3D case in Fig. 5 shows that a single transferable γ does not work for all folded proteins. This weakens the claim of a single, unified, foundational model and suggests that the PMF agreement may be partly an artifact of fitting γ.
- [Overall validation, absence of a control] No experiment is presented that removes or scrambles the ESM3 signal while keeping the GNN architecture and DSSP training term. Without such a control, the observed improvement of GBn2/GNN over GBn2/ACE cannot be attributed specifically to the distillation of evolutionary information. The improvement could arise from the GNN's ability to score local backbone geometry, independent of ESM3. A control (e.g., training with shuffled SS8 targets or with the ESM3 loss term omitted) is essential to support the paper's central attribution.
minor comments (5)
- [Abstract and Conclusions] The abstract states “robust enough to drive stable, long-timescale molecular dynamics simulations,” but the reported simulations are 500 ns for a set of small fast-folding proteins. “Long-timescale” is an overstatement relative to typical protein folding timescales; the conclusion's “6.8 µs” total is also not detailed in the main text.
- [Fig. 2 and discussion of distillation accuracy] The comparison of Schake's 87.0% correct-motif probability with ESM3's 89.2% is not apples-to-apples: ESM3 predicts SS8 from sequence only, whereas Schake uses both sequence and three-dimensional backbone coordinates. The fact that Schake is slightly worse despite having structural information is noteworthy and should be discussed.
- [Introduction, “As solvation dominates folding energetics”] This assertion is used to justify the entire training strategy, but no reference or derivation is provided. A citation or a more nuanced statement (e.g., solvation is one of several competing contributions) would be appropriate.
- [Fig. 5 and free-energy profiles] The free-energy profiles are shown without error bars or a description of the statistical uncertainty. Since the comparisons are central to the paper's quantitative claims, an estimate of uncertainty would strengthen the presentation.
- [Methods, “Data Availability”] The code link is given as a GitHub URL, but the text says it “will be made available.” For a computational paper, providing a working link and a version/DOI at submission time would improve reproducibility.
Circularity Check
The one-state energy is defined with the reference structure's own SS8 labels, so its native-stability result is partly self-definitional; the PMF reproduction is softened by fitting γ against the TIP3P benchmark.
specific steps
-
self definitional
[Eq. (1), Results section 'GNN-derived energies reliably distinguish folded and unfolded protein states'; Fig. 3]
"The binary variable y_i^(j) indicates which SS8 motifs are present in the reference (folded) structure, such that the resulting one-state (os) energy selectively stabilizes this reference state."
E_os is built by feeding the reference structure's own SS8 labels into y_i^(j), so the only structural signal in Eq. (1) is agreement with the reference pattern. The section's headline result — 'For all proteins, the lowest-energy structure is that with the lowest RMSD from the folded state' — is therefore a restatement of the construction rather than an independent inference. Any conformation whose SS8 pattern matches y is favored by definition, and the GNN was trained so native-like geometries reproduce those labels. The claimed ability to 'distinguish folded and unfolded states' is thus substantially encoded into the energy rather than predicted from it.
-
fitted input called prediction
[Results, 'Multi-state energy formulation enables accurate simulation...'; Fig. 5]
"The incorporation of GBn2 also introduces additional stabilization of compact, folded conformations, necessitating the use of a smaller scaling factor (γ= 0.175)... For α3D, however, all ISMs fail to produce a folded minimum. Increasing γ in the GBn2/GNN model restores the folded basin and yields close agreement with TIP3P (Fig. S9)."
The scaling factor γ is the single adjustable strength of the distilled correction term. It is selected so that GBn2/GNN agrees with the very TIP3P reference used as benchmark; when the selected value fails for α3D, γ is raised until the TIP3P profile is matched. The reported PMF agreement is therefore partly a calibration of γ against the target data, not a parameter-free prediction. This weakens the abstract's claim that the hybrid model 'accurately reproduces protein folding free-energy landscapes.'
full rationale
The distillation procedure itself is not circular: training Schake to match ESM3 SS8 logits is a supervised distillation from an external teacher, and the multi-state energy in Eq. (2) does not encode the reference labels, so it retains independent empirical content in the unfolded-basin shapes of Fig. 5. No load-bearing self-citation chain is present; ESM3, D.E. Shaw trajectories, and TIP3P references are external. However, two steps are partly circular. Eq. (1) defines E_os using the reference structure's own SS8 labels, making the 'native state is the global minimum' result substantially a construction rather than a prediction. In addition, the PMF comparison is made after choosing γ = 0.175, and when that value fails for α3D, γ is increased to restore agreement with the TIP3P benchmark; the reported reproduction of folding landscapes is thus partly a fit of the coupling constant to the target data. The IDP claim is also weakened by the paper's own admission that plain GBn2 'most-closely matches the TIP3P reference,' though this is a validity concern rather than a circular step. Overall the central multi-state model has real independent content, but the one-state 'prediction' is by construction and the PMF benchmark is softened by fitted γ, giving partial circularity at score 6.
Axiom & Free-Parameter Ledger
free parameters (3)
- gamma (dimensionless scale of GNN energy) =
2.5 (standalone); 0.175 (hybrid with GBn2); increased for alpha-3D (Fig. S9)
- Simulation temperature T =
300 K for E_os^GNN evaluation and stability runs; 350 K for IDP runs
- Minor training hyperparameters =
lambda_reg = 1e-6; epsilon = 1e-8; cutoffs 1 nm / 2.5 nm; 1:1 loss weighting; batch 50; lr 1e-3
axioms (5)
- domain assumption ESM3 SS8 logits approximate solvent-mediated folding free energies (E = -k_BT log P).
- domain assumption Backbone-only geometry (C-alpha, C, N) retains the information needed for solvent-sensitive secondary-structure prediction.
- domain assumption DSSP SS8 classes are a sufficient solvent-sensitive observable for constructing a solvation energy.
- domain assumption GBn2 continuum electrostatics and the GNN local term are additive and non-redundant.
- domain assumption Implicit-solvent simulations with ff14SB can be compared quantitatively with TIP3P references from Lindorff-Larsen et al.
invented entities (1)
-
ESM3 SS8 'effective energy' (E = -k_BT log P(SS8 | sequence)) as an implicit-solvent free-energy proxy
independent evidence
read the original abstract
Implicit solvent models (ISMs) promise to deliver the accuracy of explicit solvent simulations at a fraction of the computational cost. However, despite decades of development, their accuracy has remained insufficient for many critical applications, particularly for simulating protein folding and the behavior of intrinsically disordered proteins. Developing a transferable, data-driven ISM that overcomes the limitations of traditional analytical formulas remains a central challenge in computational chemistry. Here we address this challenge by introducing a novel strategy that distills the evolutionary information learned by a protein language model, ESM3, into a computationally efficient graph neural network (GNN). We show that this GNN potential, trained on effective energies from ESM3, is robust enough to drive stable, long-timescale molecular dynamics simulations. When combined with a standard electrostatics term, our hybrid model accurately reproduces protein folding free-energy landscapes and predicts the structural ensembles of intrinsically disordered proteins. This approach yields a single, unified model that is transferable across both folded and disordered protein states, resolving a long-standing limitation of conventional ISMs. By successfully distilling evolutionary knowledge into a physical potential, our work delivers a foundational implicit solvent model poised to accelerate the development of predictive, large-scale simulation tools.
Figures
Reference graph
Works this paper leans on
-
[1]
P.; Zhang, B
(1) Latham, A. P.; Zhang, B. Improving Coarse-Grained Protein Force Fields with Small- Angle X-ray Scattering Data.J. Phys. Chem. B2019,123, 1026–1034. (2) Latham, A. P.; Zhang, B. Maximum Entropy Optimized Force Field for Intrinsically Disordered Proteins.J. Chem. Theory Comput.2020,16, 773–781. (3) Liu, S.; Wang, C.; Latham, A. P.; Ding, X.; Zhang, B. O...
2020
-
[8]
T.; Meuwly, M
(86) Unke, O. T.; Meuwly, M. PhysNet: A Neural Network for Predicting Energies, Forces, Dipole Moments, and Partial Charges.J. Chem. Theory Comput.2019,15, 3678–
2019
-
[94]
Operator forces for coarse-grained molecular dynamics.J
(83) Klein, L.; Kelkar, A.; Durumeric, A.; Chen, Y.; Clementi, C.; No´ e, F. Operator forces for coarse-grained molecular dynamics.J. Chem. Phys.2025,163, 104111. (84) Plainer, M.; Wu, H.; Klein, L.; G¨ unnemann, S.; No´ e, F. Consistent Sampling and Simulation: Molecular Dynamics with Energy-Based Diffusion Models.arXiv2025, arXiv:2506.17139, accessed on...
arXiv 2025
-
[313]
P.; Kornbluth, M.; Moli- nari, N.; Smidt, T
(102) Batzner, S.; Musaelian, A.; Sun, L.; Geiger, M.; Mailoa, J. P.; Kornbluth, M.; Moli- nari, N.; Smidt, T. E.; Kozinsky, B. E(3)-equivariant graph neural networks for data- efficient and accurate interatomic potentials.Nat. Commun.2022,13,
2022
-
[579]
(105) Batatia, I.; Kov´ acs, D. P.; Simm, G. N. C.; Ortner, C.; Cs´ anyi, G. MACE: Higher Order Equivariant Message Passing Neural Networks for Fast and Accurate Force Fields.arXiv2023, arXiv:2206.07697, accessed on 2024-04-22. (106) Mann, E. L.; Wagen, C. C.; Vandezande, J. E.; Wagen, A. M.; Schneider, S. C. Egret-1: Pretrained Neural Network Potentials ...
Pith/arXiv arXiv 2024
-
[2024]
H.; Mao, Y.; Pu, J.; Mei, Y.; Shao, Y
(42) Yao, S.; Van, R.; Pan, X.; Park, J. H.; Mao, Y.; Pu, J.; Mei, Y.; Shao, Y. Machine learning based implicit solvent model for aqueous-solution alanine dipeptide molecular dynamics simulations.RSC Adv.2023,13, 4565–4577. (43) Wang, J.; Olsson, S.; Wehmeyer, C.; P´ erez, A.; Charron, N. E.; de Fabritiis, G.; No´ e, F.; Clementi, C. Machine Learning of C...
Pith/arXiv arXiv 2023
-
[2453]
P.; Musaelian, A.; Simm, G
33 (103) Batatia, I.; Batzner, S.; Kov´ acs, D. P.; Musaelian, A.; Simm, G. N. C.; Drautz, R.; Ortner, C.; Kozinsky, B.; Cs´ anyi, G. The design space of E(3)-equivariant atom- centred interatomic potentials.Nat. Mach. Intell.2025,7, 56–67. (104) Musaelian, A.; Batzner, S.; Johansson, A.; Sun, L.; Owen, C. J.; Kornbluth, M.; Kozin- sky, B. Learning local ...
2025
-
[3693]
Directional Message Passing for Molecular Graphs.arXiv2022, arXiv:2003.03123, accessed on 2023-07-13
(87) Gasteiger, J.; Groß, J.; G¨ unnemann, S. Directional Message Passing for Molecular Graphs.arXiv2022, arXiv:2003.03123, accessed on 2023-07-13. (88) Gasteiger, J.; Becker, F.; G¨ unnemann, S. GemNet: Universal Directional Graph Neu- ral Networks for Molecules.arXiv2022, arXiv:2106.08903, accessed on 2023-07-13. (89) Anderson, B.; Hy, T.-S.; Kondor, R....
Pith/arXiv arXiv 2003
-
[3887]
E.; Tkatchenko, A.; M¨ uller, K.-R
(55) Chmiela, S.; Sauceda, H. E.; Tkatchenko, A.; M¨ uller, K.-R. InMachine Learning Meets Quantum Physics; Sch¨ utt, K. T., Chmiela, S., von Lilienfeld, O. A., Tkatchenko, A., Tsuda, K., M¨ uller, K.-R., Eds.; Springer International Publishing, 2020; pp 129–154. (56) Duschatko, B. R.; Fu, X.; Owen, C.; Xie, Y.; Musaelian, A.; Jaakkola, T.; Kozinsky, B. T...
Pith/arXiv arXiv 2020
-
[5739]
Development of Coarse- Grained Lipid Force Fields Based on a Graph Neural Network.J
(76) Liao, Z.; Si, T.; Wang, T.; Kai, J.-J.; Chipot, C.; Fan, J. Development of Coarse- Grained Lipid Force Fields Based on a Graph Neural Network.J. Chem. Theory Comput.2025,21, 9175–9185. 30 (77) Riveros, I.; Zhang, B. NEAT-DNA: A Chemically Accurate, Sequence- Dependent Coarse-Grained Model for Large-Scale DNA Simulations.bioRxiv2025, bioRxiv:2025.11.0...
2025
-
[9898]
M.; Isayev, O
(64) Anstine, D. M.; Isayev, O. Machine Learning Interatomic Potentials and Long-Range Physics.J. Phys. Chem. A2023,127, 2417–2431. (65) Cheng, Z.; Bi, H.; Liu, S.; Chen, J.; Misquitta, A. J.; Yu, K. Developing a Differentiable Long-Range Force Field for Proteins with E(3) Neural Network-Predicted Asymptotic Parameters.J. Chem. Theory Comput.2024,20, 5598...
2024
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.