Pith. sign in

REVIEW 4 major objections 6 minor 23 references

Unsupervised dual rewards—spectral consensus and local entropy—align 3D atomic generators better than supervised coordinate matching.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-31 03:52 UTC pith:33UPTVLT

load-bearing objection Real method for unsupervised atomic alignment, but main-text vs appendix numbers conflict and the spectral reward can reinforce a wrong mode. the 4 major comments →

arxiv 2607.28553 v1 pith:33UPTVLT submitted 2026-07-30 cs.LG cs.AIcs.MA

APO: Unsupervised Atomic Policy Optimization for 3D Structure Prediction of Atomic Systems

Shentong Mo , Yatao Bian This is my paper
classification cs.LG cs.AIcs.MA
keywords 3D structure predictionflow matchingunsupervised alignmentgroup-relative policy optimizationspectral consistencycrystal entropyantibody CDRmaterials discovery
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper claims that flow-matching models for crystal and antibody 3D structures can be aligned without any ground-truth coordinates. Instead of ranking samples by distance to experimental labels, APO draws groups of candidates and scores them with two intrinsic signals: how strongly each lies on the dominant mode of the group’s own similarity matrix, and how low its local spatial entropy is. On standard crystal and antibody benchmarks the unsupervised loop beats the supervised preference baseline in match rate and RMSD, and also straightens the generative paths so inference needs fewer steps. A sympathetic reader cares because experimental structures for novel materials and de novo proteins are scarce or nonexistent; if intrinsic physical consistency is enough, the labeled-supervision bottleneck disappears.

Core claim

APO shows that fully unsupervised alignment of flow-matching policies—driven only by a spectral consistency score (projection onto the leading eigenvector of a group similarity matrix) plus a crystal/local-density entropy proxy—consistently outperforms supervised preference optimization that uses ground-truth RMSD. The same dual-reward group-relative update raises match rates on Perov-5, MP-20 and MPTS-52 and lowers CDR-loop RMSD on SAbDab, while straightening probability paths.

What carries the argument

The dual-reward group-relative advantage: for each group of G generated structures the Spectral Consistency Score rewards samples aligned with the principal eigenvector of their equivariant-feature similarity matrix, while the Crystal Entropy Proxy penalizes high local-density Shannon entropy; advantages are standardized inside the group and reweight the flow-matching path integral, with no critic and no ground-truth labels.

Load-bearing premise

The method assumes that the structure the current generator most agrees on inside a random sample group is the physically correct one, rather than a shared hallucination or collapsed mode.

What would settle it

Start APO from a deliberately weak or mode-collapsed base generator on MP-20; if the unsupervised loop fails to raise match rate above the base (or drifts farther from true lattices) while supervised DPO still improves, the claim that policy consensus is a valid physical proxy is falsified.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Novel crystal phases and de novo proteins can be refined without experimental coordinate labels.
  • Intrinsic spectral-plus-entropy rewards can replace noisy RMSD supervision in atomic flow matching.
  • OT/VP/VE probability paths straighten under APO, cutting inference cost.
  • Equivariant policies stay E(3)-consistent after the unsupervised update.
  • Group tournaments act as on-the-fly ensemble refinement for multi-modal loops such as CDR-H3.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same spectral-consensus filter could serve as a label-free regularizer for other geometric generators (ligands, RNA) where energy evaluators are expensive.
  • If the leading-eigenvector proxy is fragile early in training, a curriculum that gradually up-weights spectral reward may be required; the paper does not test this.
  • Path straightening suggests unsupervised physical rewards can recover near-optimal-transport maps without solving the OT problem explicitly.
  • On genuinely multi-stable systems (polymorphs) consensus may collapse the diversity the method still wants for antibody design—an open failure mode.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The manuscript proposes APO (Atomic Policy Optimization), a fully unsupervised alignment method for flow-matching generators of 3D atomic structures. It adapts group-relative policy optimization (GRPO) with a dual intrinsic reward: (i) a Spectral Consistency Score R_spec from the leading eigenvector of a group self-similarity matrix over equivariant embeddings (Eqs. 2–3), and (ii) a Crystal Entropy Proxy R_phys based on a local-density Shannon entropy (Eqs. 4–5). The policy is updated via group-normalized advantages (Eq. 6) and an advantage-weighted flow-matching objective with KL regularization (Eqs. 7–8). The central empirical claim is that APO, without any ground-truth coordinates at alignment time, consistently outperforms the supervised FlowDPO baseline on Perov-5/MP-20/MPTS-52 match rate and RMSE (Table 1) and on SAbDab CDR L1–H3 Cα/backbone RMSD (Table 2), while also straightening probability paths. Several propositions (3.1–3.6) are offered to justify the rewards and the update.

Significance. If the empirical claim holds under a fixed, reproducible protocol, the work is significant for AI-for-science: unsupervised preference-style alignment of geometric generative models would reduce dependence on expensive experimental labels for novel crystals and de novo biomolecules. Adapting GRPO to E(3)-equivariant flow matching with lightweight geometric/thermodynamic proxies is a concrete and timely contribution. The dual-task evaluation (materials + antibodies), path-ablation tables, and explicit discussion of path straightening are strengths. The theoretical propositions are only sketch-level and do not constitute machine-checked proofs; the value of the paper therefore rests primarily on the consistency and reproducibility of the SOTA tables and on whether the intrinsic rewards are shown not to reinforce unphysical modes of the base policy.

major comments (4)
  1. [Table 1; §4.1; Appendix Table 5–6] Main text vs appendix are inconsistent on the quantities needed to evaluate the central SOTA claim. Table 1 reports base OT+VE Match Rate on MP-20 as 58.94% and Full APO as 63.05%; appendix Table 6 reports Base Flow Model 42.5% and Full APO 68.7%. §4.1 sets G=8 and LR=1e-4; appendix Table 5 sets G=16 and LR=1e-5. These discrepancies make it impossible to verify that unsupervised APO reliably beats supervised FlowDPO. All main and appendix tables, hyperparameters, and base checkpoints must be reconciled to a single fixed protocol before the claim can be assessed.
  2. [§3.2 Eqs. (4)–(5); Appendix C.1 Eq. (10)] R_phys is defined differently in the main text and appendix. Main Eq. (5) is the negative Shannon entropy of a Gaussian local-density estimator ρ_j; appendix Eq. (10) is a pairwise 1/||r_i−r_j||² clash term plus a lattice regularizer. These are not equivalent. The manuscript must commit to one formula, state it once, and re-run (or clearly re-identify) all experiments under that definition. Until then, the dual-reward mechanism and the ablation in Table 3/Table 6 are not interpretable.
  3. [§3.2 Prop. 3.1; Eqs. (2)–(3); Fig. 1] Proposition 3.1 only argues that u_1 tracks the dominant cluster of the current policy’s latent samples; it does not establish that this cluster lies on the physical manifold. Figure 1 and §1 explicitly note that the base generator can place mass on metastable/hallucinated minima. If the high-density mode is unphysical, maximizing R_spec (Eq. 3) and the group advantage (Eq. 6) amplify that error. The paper needs either (a) empirical diagnostics (e.g., energy/validity of the principal mode before vs after APO, mode-collapse metrics, failure cases where consensus is wrong) or (b) a concrete condition under which the dominant mode is guaranteed to be physical. Without this, the unsupervised “self-correction” claim is not load-bearing.
  4. [§3.2 Prop. 3.3] Proposition 3.3 assumes that R_total is monotonically decreasing in RMSD(x̂, x_0) and concludes that ∇L_APO and ∇L_DPO share a descent half-space. That monotonicity is neither proved nor validated (no scatter of R_spec/R_phys/R_total vs RMSD on held-out samples is shown). If the assumption fails for the actual reward weights (α, β), the comparison to FlowDPO does not follow. A short empirical check of rank correlation between R_total and RMSD (and of pairwise preference agreement with RMSD ordering) on the evaluation sets is required to support Prop. 3.3 and the “superior to supervised coordinate matching” narrative.
minor comments (6)
  1. [Abstract] Abstract and title truncate reward names (“a that reinforces”, “a that enforces”); restore full names (Spectral Consistency Score, Crystal Entropy Proxy) for readability.
  2. [§3.3 Eqs. (7)–(8)] Eq. (7) writes an objective containing ∇_θ log π_θ inside the loss being differentiated; clarify whether this is a notational stand-in for the REINFORCE-style surrogate or an actual path-integral gradient estimator, and align with Eq. (8).
  3. [§4.3; Table 3] Table 3 caption refers to “Table 6” in the prose of §4.3 (“As shown in Table 6”); fix cross-references between main-text and appendix ablation tables.
  4. [§3.2 Props. 3.1–3.2] Proposition proof sketches invoke Perron–Frobenius on S and Gibbs free-energy arguments without stating assumptions on the kernel or the constant-potential regime; a short remark on when they fail would help.
  5. [§4.3; Appendix E Fig. 5] Report wall-clock or NFE evidence for the claimed path-straightening / inference-efficiency gain; Fig. 5 is qualitative PCA only.
  6. [§2; §3; Appendix A–B] Minor typos: “V AEs”, “de novoproteins”, “aspectral”, “acrystal”; standardize notation for reward weights (α,β vs λ_spec, λ_phys vs λ_1, λ_2).

Circularity Check

2 steps flagged

Theoretical props 3.1 and 3.3 are largely tautological (dominant mode assumed = target; reward assumed monotone in RMSD ⇒ same half-space as DPO); empirical SOTA claims are externally benchmarked and not forced by construction.

specific steps
  1. self definitional [§3.2 Proposition 3.1 and Eq. (3)]
    "If the latent embeddings {zi}G_i=1 are sampled i.i.d. from a mixture of a target structural mode and Gaussian noise, then the principal eigenvector u1 of the similarity matrix S converges to the direction of the maximum likelihood estimate of the dominant structural mode as G→∞. ... Under the assumption that the true physical structure constitutes the primary cluster in the policy’s manifold, the Perron-Frobenius theorem guarantees that the lead eigenvector u1 represents the central density of this cluster. Thus, maximizing Rspec(x̂i)=⟨zi,u1⟩ is equivalent to aligning the sample with the polic"

    The claim that R_spec aligns samples with the physical manifold is obtained by assuming the sampling distribution is already a mixture whose primary cluster is the target physical mode, then noting that the leading eigenvector tracks that primary cluster by construction of spectral clustering. The ‘derivation’ therefore inserts the desired conclusion (dominant mode = physical target) as a premise rather than establishing it; R_spec only recovers policy consensus, which equals the physical structure only by the baked-in assumption.

  2. self definitional [§3.2 Proposition 3.3]
    "Let ∇θ L_DPO be the supervised gradient from FlowDPO using ground-truth x0. Under the condition that R_total(x̂) is monotonically decreasing with respect to RMSD(x̂,x0), the unsupervised APO gradient ∇θ L_APO points in the same descent half-space as ∇θ L_DPO. ... If R_total is a valid proxy for structural similarity, then for any pair (xw,xl) where RMSD(xw,x0)<RMSD(xl,x0), we have Rw>Rl and consequently Aw>Al. Since the APO objective maximizes the log-probability of samples with higher Ai, the resulting parameter update Δθ reinforces the same probability transitions as the supervised DPO loss,"

    Gradient half-space equivalence is not derived from the intrinsic rewards; it is conditioned on R_total already ranking samples exactly as RMSD-to-ground-truth does. Under that condition the conclusion is immediate from the definitions of advantage-weighted policy gradient and DPO preference pairs. The proposition therefore renames the assumption ‘R is a valid RMSD proxy’ as a theorem that unsupervised and supervised updates agree, without independent content showing the spectral/entropy rewards satisfy the monotonicity premise.

full rationale

APO’s central empirical claim—that unsupervised dual-reward GRPO beats supervised FlowDPO on Perov-5/MP-20/MPTS-52 and SAbDab—is evaluated against held-out experimental structures and is not a fitted-input-called-prediction or a self-citation chain. Authors do not import a uniqueness theorem from their own prior work; FlowDPO/GRPO citations are external. The spectral reward is intentionally self-referential (group consensus), which is a validity risk if the base policy’s dominant mode is unphysical, but that is not Pith-circularity: nothing in the training objective is algebraically identical to the reported match-rate/RMSD metrics. Two load-bearing theory steps are thin/tautological: Prop. 3.1 builds “target structural mode is the primary cluster” into the sampling assumption then concludes the leading eigenvector tracks the target; Prop. 3.3 assumes R_total is monotone in RMSD(x̂,x0) then concludes APO and DPO gradients share a half-space—i.e., “if the proxy ranks like the supervised metric, optimizing it is like supervised optimization.” Those steps inflate the apparent first-principles support for the rewards without forcing the tables. Internal main-text vs appendix conflicts (base MRs, G, LR, R_phys formula) are reproducibility/correctness issues, not circular derivation. Score 3 reflects mild self-definitional padding in the propositions only.

Axiom & Free-Parameter Ledger

6 free parameters · 6 axioms · 3 invented entities

The central claim rests on a pretrained flow-matching policy already near the data manifold, on the identification of ‘correct structure’ with the leading spectral mode of a finite sample group, on a geometric entropy proxy standing in for thermodynamic stability, and on several hand-chosen scalars (G, τ, σ, reward weights, KL coefficient). Propositions supply motivation rather than closed derivations. No new physical constants are fitted to external experiments; the free parameters are algorithmic.

free parameters (6)
  • group_size_G = 8 (main) / 16 (appendix)
    Number of samples per tournament; controls stability of the leading eigenvector and variance of advantages. Main text sets 8; appendix sets 16.
  • spectral_temperature_tau = 0.1
    Softmax temperature in the similarity matrix S (Eq. 2); chosen by hand.
  • entropy_kernel_sigma = 0.2
    Gaussian width for local density ρj (Eq. 4); directly shapes R_phys.
  • reward_weights_alpha_beta = α=1.0, β=0.5 (appendix)
    Relative weighting of R_spec vs R_phys in total reward; appendix lists λ_spec=1.0, λ_phys=0.5.
  • KL_penalty_eta_or_beta = 0.01
    Strength of trust-region pull toward the reference flow; appendix β=0.01.
  • learning_rate = 1e-4 (main) / 1e-5 (appendix)
    AdamW step size for alignment; inconsistent across sections.
axioms (6)
  • ad hoc to paper The physically correct structure coincides with the dominant latent mode of the current policy’s sample group, so the leading eigenvector of S is a valid unsupervised target (Prop. 3.1).
    Load-bearing for R_spec; fails if the pretrained policy’s primary cluster is a hallucination or metastable mode.
  • domain assumption Local-density Shannon entropy (or the appendix pairwise repulsion form) is an upper bound / proxy for the configurational part of Gibbs free energy under roughly constant potential (Prop. 3.2).
    Standard thermodynamic intuition, but the specific differentiable surrogate is a modeling choice not validated against DFT/MD energies in the paper.
  • ad hoc to paper R_total is monotonically decreasing in RMSD to ground truth, hence APO gradients share a descent half-space with FlowDPO (Prop. 3.3).
    Assumed rather than measured; used to claim unsupervised–supervised gradient equivalence.
  • standard math Group-relative advantages with mean baseline remain unbiased policy-gradient estimators for the flow-matching path-integral log-likelihood approximation (Prop. 3.4, Eqs. 6–8).
    Standard baseline argument from policy gradients; the path-integral approximation of ∇log π for flows is taken from the FM literature.
  • domain assumption A pretrained flow-matching model on labeled crystals/antibodies already places non-negligible mass near physical structures, so self-correction can improve rather than collapse the policy.
    Implicit in all experiments; APO is alignment, not training from scratch without labels.
  • standard math Equivariant backbone features and E(3)-invariant distances make advantages invariant and preserve equivariance of the updated flow (Prop. 3.6).
    Standard equivariance composition argument if the architecture and rewards truly have those symmetries.
invented entities (3)
  • Spectral Consistency Score (R_spec) no independent evidence
    purpose: Provide a label-free group reward by projecting each sample embedding onto the leading eigenvector of the group similarity matrix.
    Core unsupervised signal; defined in Eqs. 2–3. No external validation that high R_spec predicts low energy or experimental match beyond the paper’s own tables.
  • Crystal Entropy Proxy (R_phys) no independent evidence
    purpose: Differentiable thermodynamic-stability surrogate from local atomic density entropy (or pairwise repulsion in the appendix).
    Second half of the dual reward; formula disagrees between main text and appendix, and is not calibrated to measured free energies.
  • APO objective (group-relative advantage-weighted flow-matching update) no independent evidence
    purpose: Replace supervised DPO pairs with intrinsic group advantages plus KL to a reference flow.
    Adaptation of GRPO to FM vector fields for atomic coordinates; the combination is the paper’s method entity.

pith-pipeline@v1.2.0-daily-grok45 · 17945 in / 4873 out tokens · 85849 ms · 2026-07-31T03:52:32.550199+00:00 · methodology

0 comments
read the original abstract

Predicting the 3D structures of atomic systems is fundamental to advancing material science and drug discovery. While flow-matching models (, FlowDPO) have recently shown promise in this domain, their performance relies heavily on alignment with ground-truth coordinates via supervised preference learning. However, obtaining experimental labels for novel crystal phases or de novo proteins is prohibitively expensive, creating a bottleneck for structural modeling in data-scarce regimes. In this work, we propose (Atomic Policy Optimization), a fully unsupervised alignment framework that eliminates the need for ground-truth reference structures. APO adapts group-relative policy optimization to 3D atomic environments, utilizing a novel dual-reward mechanism: (i) a that reinforces the policy's dominant latent structural modes through eigen-decomposition of sample similarities, and (ii) a that enforces thermodynamic stability. Our framework enables the model to ``self-correct'' by identifying physically plausible configurations within sampled groups. Extensive benchmarks on crystal and antibody structure prediction demonstrate that APO consistently outperforms fully supervised baselines, achieving a new state-of-the-art in match rates and structural fidelity. Furthermore, we show that APO effectively straightens probability paths, significantly improving inference efficiency. Our results suggest that intrinsic physical consistency can serve as a superior guide for alignment compared to noisy, supervised coordinate matching.

Figures

Figures reproduced from arXiv: 2607.28553 by Shentong Mo, Yatao Bian.

Figure 1
Figure 1. Figure 1: 3D representation of the atomic energy landscape, highlighting the challenge of local minima [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Illustration of the proposed APO framework for 3D structure prediction. APO enables unsuper [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: A quantitative line graph (based on our experimental discussion) showing the relationship between [PITH_FULL_IMAGE:figures/full_fig_p014_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Evolution of Crystal Lattices. Comparison of generated structures for generic composition AB2O4 (Spinels). (Top) Base model samples show distorted lattices and overlapping atoms. (Bottom) APO-aligned samples exhibit clear cubic symmetry and correct atomic spacing, solely driven by intrinsic rewards. E Qualitative Visualizations In this section, we provide a visual analysis of the structural quality and the… view at source ↗
Figure 5
Figure 5. Figure 5: Path Straightening Visualization. 2D PCA projection of the generation trajectory. (Left) Supervised Fine-Tuning often results in curved paths as it forces the model to memorize specific coordinates. (Right) APO results in linear trajectories, indicating a more direct mapping from noise to the structural manifold. F Discussions Limitations. Despite its success, several avenues for improvement remain. First,… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

23 extracted references · 3 linked inside Pith

  1. [1]

    Highly accurate protein structure prediction with alphafold.Nature, 595(7868):583–589, 2021

    John Jumper, Richard Evans, Alexander Pritzel, Tim Green, Michael Figurnov, et al. Highly accurate protein structure prediction with alphafold.Nature, 595(7868):583–589, 2021

    2021

  2. [2]

    Crystal diffusion variational autoencoder for periodic material generation

    Tian Xie, Xiang Fu, Octavian-Eugen Ganea, Regina Barzilay, and Tommi Jaakkola. Crystal diffusion variational autoencoder for periodic material generation. InInternational Conference on Learning Representations (ICLR), 2022

    2022

  3. [3]

    Torsional diffusion for molecular conformer generation

    Bowen Jing, Gabriele Corso, Jeffrey Chang, Regina Barzilay, and Tommi Jaakkola. Torsional diffusion for molecular conformer generation. InAdvances in Neural Information Processing Systems (NeurIPS), 2022

    2022

  4. [4]

    Antigen-specific antibody design and optimization with diffusion-based generative models

    Shitong Luo, Yufeng Su, Jian Peng, and Jianzhu Ma. Antigen-specific antibody design and optimization with diffusion-based generative models. InAdvances in Neural Information Processing Systems (NeurIPS), 2022

    2022

  5. [5]

    Yaron Lipman, Ricky T. Q. Chen, Heli Locus, Patrick Esser, and Mattia Le. Flow matching for scalable generative modeling. InInternational Conference on Learning Representations (ICLR), 2023

    2023

  6. [6]

    Building normalizing flows with stochastic interpolants

    Michael S Albergo and Eric Vanden-Eijnden. Building normalizing flows with stochastic interpolants. InInternational Conference on Learning Representations (ICLR), 2023

    2023

  7. [7]

    Denoising diffusion probabilistic models.Advances in Neural Information Processing Systems (NeurIPS), 2020

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models.Advances in Neural Information Processing Systems (NeurIPS), 2020

    2020

  8. [8]

    3d structure prediction of atomic systems with flow-based direct preference optimization

    Rui Jiao, Xiangzhe Kong, Wenbing Huang, and Yang Liu. 3d structure prediction of atomic systems with flow-based direct preference optimization. InAdvances in Neural Information Processing Systems (NeurIPS), 2024

    2024

  9. [9]

    Direct preference optimization: Your language model is secretly a reward model

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. InAdvances in Neural Information Processing Systems (NeurIPS), 2023

    2023

  10. [10]

    Commentary: The materials project: A collaborative ecosystem for accelerating materials design.APL Materials, 1(1):011002, 2013

    Anubhav Jain, Shyue Ping Ong, Geoffroy Hautier, Wei Chen, William Davidson Richards, et al. Commentary: The materials project: A collaborative ecosystem for accelerating materials design.APL Materials, 1(1):011002, 2013

    2013

  11. [11]

    The protein data bank.Nucleic Acids Research, 28(1):235–242, 2000

    Helen M Berman, John Westbrook, Zukang Feng, Gary Gilliland, TN Bhat, Helge Weissig, Ilya Y Shindyalov, and Philip E Bourne. The protein data bank.Nucleic Acids Research, 28(1):235–242, 2000

    2000

  12. [12]

    De novo design of protein structure and function with rfdiffusion.Nature, 620(7976):1089–1100, 2023

    Joseph L Watson, David Juergens, Nathaniel R Bennett, Brian J Ellis, Alicey Yi, et al. De novo design of protein structure and function with rfdiffusion.Nature, 620(7976):1089–1100, 2023

    2023

  13. [13]

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y . K. Li, Y . Wu, and Daya Guo. Deepseekmath: Pushing the limits of mathematical reasoning in open language models.arXiv preprint arXiv:2402.03300, 2024

    Pith/arXiv arXiv 2024

  14. [14]

    Protein structure generation via folding-diffusion.arXiv preprint arXiv:2205.01561, 2022

    Magnus AE Hoffmann, Harrison Edwards, Jianzhu Ma, Jian Peng, Katherine Heller, and Yufeng Su. Protein structure generation via folding-diffusion.arXiv preprint arXiv:2205.01561, 2022

    Pith/arXiv arXiv 2022

  15. [15]

    Predicting crystal structures via self-equivariant graph diffusion.arXiv preprint arXiv:2303.03543, 2023

    Rui Jiao, Wenbing Huang, Peijia Lin, Jiaqi Han, Bohan Chen, Yi Luo, Yang Liu, and Jian Peng. Predicting crystal structures via self-equivariant graph diffusion.arXiv preprint arXiv:2303.03543, 2023

    Pith/arXiv arXiv 2023

  16. [16]

    Deep reinforcement learning from human preferences

    Paul F Christiano, Jan Leike, Tom Brown, Milos Martic, Shane Legg, and Dario Amodei. Deep reinforcement learning from human preferences. InAdvances in Neural Information Processing Systems (NeurIPS), 2017

    2017

  17. [17]

    Maziar Raissi, Paris Perdikaris, and George E Karniadakis. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations.Journal of Computational Physics, 2019. 10

    2019

  18. [18]

    A tutorial on spectral clustering.Statistics and computing, 2007

    Ulrike V on Luxburg. A tutorial on spectral clustering.Statistics and computing, 2007

    2007

  19. [19]

    Crystal structure prediction using ab initio evolutionary algorithms: Principles and applications.The Journal of Chemical Physics, 2006

    Artem R Oganov and Colin W Glass. Crystal structure prediction using ab initio evolutionary algorithms: Principles and applications.The Journal of Chemical Physics, 2006. 11 Appendix In this appendix, we provide the following material: • Additional implementation and dataset details in Section A, • The complete algorithm for APO in Section B, • Theoretica...

    2006

  20. [20]

    , x(G)} ∼πθ(·|c)

    Sampling: SampleGtrajectories{x (1), . . . , x(G)} ∼πθ(·|c)

  21. [21]

    ComputeR (i) spec via eigen-decomposition ofS(Eq

    Compute Intrinsic Rewards: Compute Similarity MatrixS ij =Sim(f(x (i)), f(x(j))). ComputeR (i) spec via eigen-decomposition ofS(Eq. 9). ComputeR (i) phys via spatial entropy (Eq. 10). Total RewardR (i) =λ 1R(i) spec +λ 2R(i) phys

  22. [22]

    Compute std devσ R = q 1 G PG k=1(R(k) − ¯R)2

    Group-Relative Advantage: Compute mean reward ¯R= 1 G PG k=1 R(k). Compute std devσ R = q 1 G PG k=1(R(k) − ¯R)2. AdvantageA (i) = R(i)− ¯R σR+ϵ . end for

  23. [23]

    centrality

    Policy Update: LAPO =− 1 G PG i=1 A(i) logπ θ(x(i)|c). Updateθ←θ−η∇ θLAPO. untilconvergence C More Discussions on APO C.1 Theoretical Motivation for Intrinsic Rewards Spectral Consistency as Manifold Learning.The Spectral Consistency Score ( Rspec) is grounded in spectral graph theory. By constructing a similarity graph of the generated group samples, the...