{"id":"41c5498b-352f-470b-90dd-7638e2f47cf7","arxiv_id":"2509.08734","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Recasting EquiformerV2 as a deep equilibrium model with warm-started fixed points gives faster and often more accurate force predictions on standard MD benchmarks, though energy accuracy on OC20 is worse.","lead":"The authors turn the EquiformerV2 force-field network into a deep equilibrium model, reusing features from the previous simulation step to speed up inference. On MD17, MD22 and OC20 200k they report faster inference and better force accuracy, with lower memory use and fewer parameters.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Fixed-point convergence is only spot-checked on Aspirin, yet the inference protocol's relaxed tolerance and 40-step cap could make the DEQ output ill-defined on other systems, which would undermine the speed and accuracy claims.","rationale":"The reader identified the unverified fixed-point convergence as the weakest assumption, and my reading agrees. The paper provides no proof of convergence for Anderson acceleration on this equivariant architecture, only an Aspirin example. Since the DEQ output is defined as the fixed point and the IFT gradient assumes that fixed point is reached, any non-convergence breaks both the forward and backward passes. The relaxed inference tolerance makes this concern concrete: the solver deliberately stops at a residual of 1e-1, and with a 40-step cap there is no guarantee that even that loose residual is reached on all systems. The OC20 solver-step statistics (mean 11-29, std up to 7) show that the cap is not far away for some samples. The Markov-property test averages deviations below 1%, which could mask rare but severe non-convergence. The paper deserves credit for stability checks on Aspirin, scaling experiments, and the fixed-point correction loss, but those do not cover all molecules and training stages. Because this concern is exactly the one the reader flagged, the CONDITIONAL verdict remains appropriate. A quick diagnostic on the released model would either confirm convergence across systems (strengthening the paper) or reveal a real limitation that must be documented.","tokens_in":21617,"tokens_out":5059,"duration_ms":39422,"concrete_test":"Run the released DEQuiformer with the paper's exact inference settings (warm start from the previous step, epsilon=1e-1, maximum 40 Anderson steps) on all MD17/MD22 molecules and at least 1000 OC20 validation samples. Record for each sample the residual ||f_theta(h)-h||/||h|| at termination and whether the 40-step cap was hit. Then separately compute force/energy errors for samples that hit the cap or have residual above 1e-1, and compare with samples that converged. If the non-converged fraction exceeds 1% on any molecule, or if the non-converged errors are more than 2x the converged errors, the fixed-point assumption fails and the reported speed and accuracy gains are not supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim rests on the DEQ forward pass being a well-defined fixed point of the Anderson-accelerated solver: h* = f_theta(h*, x), with the IFT gradient in Eq. 9 depending on that exact fixed point. No contraction or Lipschitz condition is stated for f_theta, and Anderson acceleration has no general convergence guarantee. The paper spot-checks fixed-point convergence only on Aspirin (Fig. 4a) and the tolerance-sensitivity curve only on Aspirin (Fig. 2a). During inference, the solver stops at a relaxed tolerance of epsilon=1e-1 after warm-starting from the previous time step, and the pseudocode caps the solver at 40 steps. On OC20 relaxation, the average number of solver steps is 11-29 with standard deviations of 2-7, so a non-negligible fraction of samples may hit the 40-step cap and return a non-equilibrium iterate. If the fixed point is not reached, the output is not the DEQ equilibrium, the IFT gradient is invalid, and the reported speedups (which rely on loose stopping) may come at an unmeasured accuracy cost on other molecules. The Markov-property check also averages over samples and could hide a small fraction of badly non-converged states. This is the deepest assumption because it underpins both the correctness of training and the validity of every reported accuracy and speed number.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes DEQuiformer, a deep equilibrium variant of EquiformerV2 for machine-learned force fields. The deep stack of equivariant transformer layers is replaced by a weight-tied block whose output is defined as a fixed point of an Anderson-accelerated solver, trained with implicit-function-theorem gradients. The main idea is to exploit the temporal continuity of molecular dynamics by warm-starting the fixed-point solver from the previous timestep and relaxing the solver tolerance at inference. The authors report improved force accuracy and 10-20% speedups on MD17, MD22, and OC20 200k relative to EquiformerV2, with up to 5x fewer parameters and reduced training memory.","tokens_in":21928,"tokens_out":9934,"duration_ms":82997,"significance":"If the claims hold, the paper identifies a useful and underused inductive bias for ML force fields—temporal continuity—and demonstrates a practical way to obtain deep-model accuracy at low memory cost. The strengths are the clean connection to SCF restart practices, the memory-efficient IFT training, and the competitive MD17/MD22 force errors at lower parameter counts. However, the significance is currently undercut by overclaimed OC20 numbers and by the absence of fixed-point convergence diagnostics outside a single molecule, so the central empirical claims are not yet fully supported.","major_comments":[{"comment":"The abstract's claim that the method 'improve[s] both accuracy and speed by 10%-20% on ... OC20 200k' is not supported by the reported numbers. In Table 1b, DEQuiformer's OC20 energy error is 0.498 eV versus 0.392 eV for EquiformerV2 (27% worse), while the force error improves from 0.038 to 0.035 eV/Å (about 8%). The caption of Table 1b states that DEQuiformer is 'more accurate than EquiformerV2 on forces and energy', which is directly contradicted by the same table. The OC20 relaxation speedup in Table 1a is 12.38 s versus 12.92 s, approximately 4%, not 10-20%. The authors should report force and energy errors separately and restrict the 10-20% claim to the datasets and metrics where it actually holds.","section":"Abstract, §4.1, Table 1b"},{"comment":"The OC20 speed comparison and accuracy comparison use different baselines. Table 1a compares the one-layer DEQuiformer with the 14-layer EquiformerV2, whereas Table 1b reports accuracy only for the 8-layer EquiformerV2 (force 0.038 eV/Å, energy 0.392 eV); the accuracy of the 14-layer model is not reported. A speed advantage over a 14-layer model does not by itself establish a speed advantage at matched accuracy, because the 14-layer model may not be the same accuracy point as the DEQuiformer. To support the 'faster than EquiformerV2' claim, the authors should report relaxation times and accuracies for the same baseline (e.g., 8-layer) and for the 14-layer model on the same dataset.","section":"§4.1, 'Speedup in relaxations', Table 1a"},{"comment":"The DEQ output is well-defined only if the Anderson-accelerated solver converges to h* = f_theta(h*, x), and the IFT gradient in Eq. (9) is valid only at such a fixed point. The paper states no contraction or stability condition for f_theta and demonstrates convergence only on Aspirin (Fig. 4a); the tolerance-sensitivity study (Fig. 2a) is also on Aspirin only. During inference the solver stops at the relaxed tolerance epsilon_FP_reuse_test = 1e-1 with a cap of 40 forward steps (Table 3), so for other molecules or training stages the returned iterate may not be the equilibrium. The Markov-property test in Appendix A.3 reports a mean relative force deviation of 0.4% on Aspirin and 0.8% on OC20, but it compares warm-started relaxed predictions with zero-initialized relaxed predictions, not with tightly converged fixed points, so it cannot rule out a systematic bias from early stopping. Please provide per-system fixed-point residuals, the fraction of samples hitting the 40-step cap, and a comparison of warm-started relaxed predictions with tightly converged predictions across all MD17/MD22 molecules and OC20.","section":"§3.1, Eq. (9), Fig. 4a, Table 3, Appendix A.3"},{"comment":"The central quantitative claims rest on single training runs without error bars. The checklist acknowledges this, but the headline 10-20% improvements and the '5/8 best' statements in Section 4.1 have no variance estimates, and the OC20 relaxation time difference in Table 1a (12.38 ± 0.33 s vs 12.92 ± 0.26 s) is small relative to the reported deviations. At minimum, the authors should report multiple seeds for the main comparisons, or explicitly state which reported differences are within run-to-run noise.","section":"Tables 1, 2, 4; NeurIPS Checklist item 7"}],"minor_comments":[{"comment":"Typos such as 'fa,r' (Abstract), 'Immproved accuracy' (Section 4.1 heading), 'themself' (Introduction), 'variies' (Section 4), 'furhter' (Conclusion), and 'are are common practice' (Section 4.1) should be corrected.","section":"Throughout"},{"comment":"The text says the first inference timestep uses epsilon_train = 1e-4 and later timesteps use epsilon_FP_reuse_test = 1e-1, but Table 3 lists a single inference tolerance of 1e-1; please label the table entry as applying to reuse steps and state the first-step tolerance.","section":"§3.1 and Table 3"},{"comment":"The solver-step distribution without fixed-point reuse does not sum to 100% in the main figure; the appendix explains this is due to a long tail, but the main text should note the log-scale tail to avoid confusion.","section":"Figure 2b and Appendix A.3"},{"comment":"'Time is measured as the forward pass' should be clarified for Table 1a and Table 4 to state whether reported times include the whole relaxation trajectory, the DEQ solver iterations, and any integration overhead.","section":"§4 and Tables 1a, 4"},{"comment":"Since the method is a modification of a specific EquiformerV2 commit and the TorchDEQ library, please specify the exact commits and versions of all dependencies, and describe any modifications to the solver, to aid reproducibility.","section":"Appendix A.2"}],"recommendation":"major_revision","confidential_remarks":"The paper is a reasonable empirical contribution if the claims are revised and the convergence diagnostics are added. In my view the fixed-point convergence issue is the main technical risk; I would ask the authors to provide per-system residual and step-cap statistics before further consideration. I would also flag that the abstract's unified 10-20% claim is contradicted by their own OC20 energy and relaxation numbers."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"What you should know: this is the first sensible application of deep equilibrium models to equivariant ML force fields, and the fixed-point reuse across MD steps is a genuinely nice idea. The paper's headline claim, however, is broader than the evidence supports.\n\nThe new thing here is not the DEQ machinery—it's all imported from Bai et al. and the optical flow follow-up—but the application to an equivariant GNN force field, plus the warm-starting from the previous MD step's fixed point. That is a natural fit because consecutive MD configurations are nearly identical. The authors also make sensible engineering choices: Anderson acceleration, IFT gradients for memory savings, recurrent dropout, and a fixed-point correction loss. On MD17/MD22 the force errors are often lower than their re-trained EquiformerV2 baselines, and the parameter and memory reductions are real. The OC20 relaxation experiment is a good practical test.\n\nThe soft spots are mostly about claims versus data. The abstract says 'improve both accuracy and speed by 10%-20%' on MD17, MD22, and OC20 200k. That doesn't hold on OC20: the energy error is worse (0.498 vs 0.392 eV), and the relaxation speedup is about 4% (12.38s vs 12.92s). Table 1b's caption even says DEQuiformer is more accurate on both forces and energy, which is false for energy. The baselines are also self-trained reduced EquiformerV2 models (l=3, up to 14 layers), not the original published checkpoints, so the comparison is weaker than it looks.\n\nThe stress-test about fixed-point convergence is worth taking seriously but is not fatal. The paper only spot-checks convergence on Aspirin, and inference uses a relaxed tolerance of 1e-1 with a 40-step cap, so outputs are not exact equilibria. However, training uses a tight tolerance (1e-4), and the Markov-property check shows <1% deviation due to reuse. Still, the paper would be much stronger with convergence diagnostics across all systems and a sensitivity analysis on the tolerance. No error bars and no code release also limit confidence.\n\nWho this is for: anyone working on efficient ML force fields or implicit models in scientific ML. It's a solid empirical paper that should go to peer review. The revision needs to recalibrate the abstract, fix the caption, add convergence checks, and ideally release the code.","headline":"First real DEQ-for-force-fields paper with a genuinely useful fixed-point reuse trick; the 10-20% claim is too broad and the OC20 numbers don't back it up, but it deserves a serious referee.","tokens_in":22462,"tokens_out":3340,"would_cite":false,"duration_ms":388127,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"By recasting a state-of-the-art equivariant force-field network as a deep equilibrium model and warm-starting its fixed-point solver from the previous simulation step, the paper reports 10-20% gains in both accuracy and speed on the MD17…","keywords":["deep equilibrium models","machine learning force fields","molecular dynamics","equivariant graph neural networks","fixed-point reuse","implicit models","temporal continuity","EquiformerV2"],"falsifier":"Run DEQuiformer at the training tolerance ($\\epsilon = 10^{-4}$) on all MD17/MD22 molecules and log Anderson solver steps per sample across training; if any system or checkpoint fails to converge within the 40-step maximum, the model has no well-defined forward pass and the IFT gradient does not exist. A complementary test: measure the force deviation between warm-started and zero-initialized predictions on trajectories binned by inter-step displacement; if the deviation grows substantially with step size instead of staying below the reported 1%, the temporal-continuity premise that carries the speedup fails precisely where it is needed.","tokens_in":21402,"feed_emoji":"⚛️","tokens_out":12830,"duration_ms":98555,"temperature":0.7,"pith_summary":"Machine-learning force fields currently treat every molecular dynamics step as an independent prediction, ignoring that a simulation is continuous and consecutive states are nearly identical. The paper argues that this temporal continuity is usable prior information and shows how to exploit it: recast the state-of-the-art EquiformerV2 architecture as a deep equilibrium model (DEQ), in which a few weight-tied layers are iterated to a fixed point instead of being stacked into a deep explicit network. At inference, the fixed-point solver is warm-started from the previous time step's fixed point, recycling intermediate features rather than recomputing them. The authors report improving both accuracy and speed by 10-20% compared to the non-DEQ base model on MD17, MD22, and OC20 200k, with memory cost no longer growing with effective depth. If correct, this makes temporal reuse a broadly applicable axis of force-field design, complementary to the symmetry priors that drove earlier progress.","feed_headline":"Recycling features makes force fields 10-20% faster and more accurate","feed_subtitle":"A warm-started deep equilibrium force field beats the explicit-stack base model on three benchmarks.","key_machinery":"The central object is the DEQ fixed-point layer: the $L$ Equiformer transformer blocks are replaced by $L_{\\text{DEQ}} \\ll L$ weight-tied blocks $g_\\theta$, and the node features are defined as the equilibrium $h^* = g_\\theta\\!\\left((h^* + \\tilde{x})\\,\\|\\tilde{x}\\|/\\|h^* + \\tilde{x}\\|\\right)$, where $\\tilde{x}$ is the encoder embedding injected at every solver step and the norm rescaling prevents feature growth. Four supporting mechanisms carry the argument: Anderson acceleration (chosen over Broyden's method for stability) as the root solver; warm-starting from the previous time step's fixed point during inference; implicit-function-theorem gradients instead of backpropagating through solver iterations, making training memory independent of depth; and the sparse fixed-point-correction loss plus recurrent path dropout for training stability, with the inference tolerance relaxed from $10^{-4}$ to $10^{-1}$ to cut solver steps.","core_discovery":"Stated on the paper's own terms: the explicit stack of EquiformerV2 transformer blocks can be replaced by an implicit-depth layer whose output is the fixed point $h^* = f_\\theta(h^*, x)$ of one or two equivariant blocks, solved with Anderson acceleration. Because MD trajectories are integrated with time steps small relative to the fastest atomic motion, the fixed point at step $t$ is close to the one at step $t+1$, so initializing the solver with the previous fixed point cuts the average number of solver steps from 5-6 to about 3 on Aspirin and from roughly 29 to 11 in OC20 relaxations. This fixed-point reuse, combined with a relaxed test-time solver tolerance of $10^{-1}$, is what turns the DEQ from slow into 10-20% faster than EquiformerV2; without both techniques, no speedup appears. Training uses implicit-function-theorem gradients instead of backpropagation through the solver trajectory, so memory use is independent of effective depth, while a sparse fixed-point-correction loss and recurrent dropout keep training stable. The paper also reports that warm-starting shifts predicted forces by less than 1% relative to zero initialization, so the reuse approximately preserves the Markov property of the dynamics.","pith_inferences":["The scheme's structure mirrors SCF warm-starting in ab initio MD, which suggests a testable gradient the paper does not measure: the reuse speedup should scale with trajectory smoothness, helping most at very small integration steps and degrading as the time step grows.","The post-training tolerance trade-off could plausibly be made adaptive per time step, spending more solver steps exactly when the configuration has moved far from the previous fixed point, such as during bond breaking or barrier crossing.","Because the solver's step count adapts to input difficulty, the DEQ form may generalize better out of distribution, with effective depth growing for unfamiliar geometries; this could be tested directly on elevated-temperature or far-from-equilibrium trajectories.","AIMD practice goes beyond simple warm-starting to Pulay mixing and predictor-corrector extrapolation of the density, a toolbox the authors explicitly flag as future work; applying the same ideas to fixed-point initialization could plausibly cut solver steps below the observed 3 per step.","The reported energy error on OC20 (0.498 vs 0.392 eV) is worse than the base model even though the force error improves, so the paper's headline 'better accuracy' claim is cleanest read as a force-accuracy claim; a fair reader should check whether the energy gap persists on larger splits."],"forward_implications":["On MD17, DEQuiformer is on average 19% faster and 15% more accurate than EquiformerV2, and wins on 5 of 8 molecules; on MD22 it outperforms EquiformerV2 on average and fits the double-walled nanotube system that runs the 8-layer baseline out of memory.","On OC20 200k, a two-layer DEQuiformer improves force error to 0.035 eV/Å from EquiformerV2's 0.038 eV/Å at a fraction of the parameter count, and the accuracy gap persists when EquiformerV2 is scaled to 14 layers, consistent with the paper's suggestion that the fixed-point formulation itself carries an inductive bias.","In OC20 relaxation simulations, the speedup appears only when fixed-point reuse and the relaxed tolerance ($10^{-1}$) are combined, reducing the number of layer evaluations from roughly 29 to about 11 per step.","Warm-starting approximately preserves the Markov property: predicted forces deviate by less than 1% (0.4% on Aspirin, 0.8% on OC20 relaxations) from zero-initialized predictions, so the reuse does not measurably bias the dynamics.","Training memory no longer grows with effective depth, which is what allows more expressive models to be trained on larger systems that exceed the explicit-stack baseline's memory budget."],"supporting_citations":[{"why":"Supplies the DEQ framework itself: the fixed-point layer formulation, the implicit-function-theorem gradient, and recurrent dropout.","marker":"[5]"},{"why":"The base architecture being recast; provides the equivariant transformer blocks, output heads, and OC20 settings that DEQuiformer inherits.","marker":"[31]"},{"why":"Source of the three key mechanisms: warm-starting from previous fixed points, the sparse fixed-point correction loss, and the relaxed inference tolerance.","marker":"[3]"},{"why":"The Anderson acceleration method used as the root solver, chosen over Broyden's method for training stability.","marker":"[2]"},{"why":"Supplies the MD17/MD22 training loop and hyperparameters used to train both models for the comparison.","marker":"[30]"},{"why":"Introduces the MD22 dataset of larger molecules used for the scaling and accuracy benchmarks.","marker":"[15]"},{"why":"The eSCN convolution that EquiformerV2's layers are built on and that the DEQ blocks inherit as their message-passing primitive.","marker":"[35]"},{"why":"Stated source of the OC20 baseline numbers that frame DEQuiformer's accuracy comparison.","marker":"[47]"}],"fun_headline_variants":["Recycle features: DEQ force fields 10-20% faster and more accurate","Warm-start fixed points give force fields a 10-20% edge","Implicit depth recycles features: faster, more accurate MD","DEQ force fields: recycle fixed points, gain 10-20%","Warm-started DEQ makes force fields 10-20% better"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The Anderson-accelerated fixed-point solver reliably converges to a well-defined equilibrium for every molecule and at every point in training, so the forward pass exists and the implicit-function-theorem gradient is valid; convergence is verified on Aspirin only, not on the other MD17/MD22 systems or OC20.","fun_headline_variants_meta":{"raw":{"variants":["Recycle features: DEQ force fields 10-20% faster and more accurate","Warm-start fixed points give force fields a 10-20% edge","Implicit depth recycles features: faster, more accurate MD","DEQ force fields: recycle fixed points, gain 10-20%","Warm-started DEQ makes force fields 10-20% better"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000656,"raw_usage":{"total_tokens":3016,"prompt_tokens":970,"completion_tokens":2046,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":586,"completion_tokens_details":{"reasoning_tokens":1946}},"tokens_in":586,"tokens_out":2046,"duration_ms":12487,"temperature":1.0,"reasoning_tokens":1946,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T15:59:49.677623+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run DEQuiformer at the training tolerance ($\\epsilon = 10^{-4}$) on all MD17/MD22 molecules and log Anderson solver steps per sample across training; if any system or checkpoint fails to converge within the 40-step maximum, the model has no well-defined forward pass and the IFT gradient does not exist. A complementary test: measure the force deviation between warm-started and zero-initialized predictions on trajectories binned by inter-step displacement; if the deviation grows substantially with step size instead of staying below the reported 1%, the temporal-continuity premise that carries the speedup fails precisely where it is needed.","supporting_citations":[{"cited_title":"Unke, Adil Kabylda, Huziel E","cited_arxiv_id":null,"evidence_quote":"Introduces the MD22 dataset of larger molecules used for the scaling and accuracy benchmarks."},{"cited_title":"Zico Kolter, and Vladlen Koltun","cited_arxiv_id":null,"evidence_quote":"Supplies the DEQ framework itself: the fixed-point layer formulation, the implicit-function-theorem gradient, and recurrent dropout."},{"cited_title":"EquiformerV2: Improved Equivariant Transformer for Scaling to Higher-Degree Representations, March 2024","cited_arxiv_id":null,"evidence_quote":"The base architecture being recast; provides the equivariant transformer blocks, output heads, and OC20 settings that DEQuiformer inherits."},{"cited_title":"Zico Kolter","cited_arxiv_id":null,"evidence_quote":"Source of the three key mechanisms: warm-starting from previous fixed points, the sparse fixed-point correction loss, and the relaxed inference tolerance."},{"cited_title":"Anderson","cited_arxiv_id":null,"evidence_quote":"The Anderson acceleration method used as the root solver, chosen over Broyden's method for training stability."},{"cited_title":"Equiformer: Equivariant Graph Attention Transformer for 3D Atomistic Graphs, February 2023","cited_arxiv_id":null,"evidence_quote":"Supplies the MD17/MD22 training loop and hyperparameters used to train both models for the comparison."},{"cited_title":"Lawrence Zitnick","cited_arxiv_id":null,"evidence_quote":"The eSCN convolution that EquiformerV2's layers are built on and that the DEQ blocks inherit as their message-passing primitive."},{"cited_title":"Lightweight equivariant model for efficient machine learning interatomic potentials, 2024","cited_arxiv_id":null,"evidence_quote":"Stated source of the OC20 baseline numbers that frame DEQuiformer's accuracy comparison."}],"review_version":2}