Pith. sign in

REVIEW 3 major objections 5 minor 51 references

Towards Faster and More Compact Foundation Models for Molecular Property Prediction

T0 review · 3 major / 5 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Removing two interaction blocks shrinks JMP-L by 32% with minimal accuracy loss.

desk verdict Useful compression study with a real confound: fixed wall-clock budget makes the 'over-parameterized' conclusion unproven. read the letter →

arxiv 2504.19538 v1 pith:I5PP6G7A submitted 2025-04-28 cs.LG q-bio.BM

Yasir Ghunaim , Andrés Villa , Gergo Ignacz , Gyorgy Szekely , Motasem Alfarra , Bernard Ghanem This is my paper · ORCID
classification cs.LGq-bio.BM
keywords molecularpropertypredictionfoundationmodelsmodelpruningblockreductionknowledgedistillationGradCAMgraphneuralnetworksGemNet-OC
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that JMP-L, a 160-million-parameter foundation model for molecular property prediction, is over-parameterized for most downstream tasks. It claims that the later interaction blocks of the model contribute least to prediction quality, so they can be removed. Pruning the two deepest interaction blocks yields a 108-million-parameter model with a 1.3x inference speedup and only a minimal drop in accuracy. If correct, practitioners can fine-tune and deploy a substantially cheaper model without retraining from scratch.

What carries the argument

The load-bearing mechanism is a GradCAM-style relevance score: features from each interaction block are weighted by their gradient with respect to the loss, passed through a ReLU, and averaged to give an importance rank per block. The paper uses this ranking to prune blocks sequentially from the deepest end, then adjusts the first FinalMLP layer either by slicing its weights or by random initialization. A short knowledge-distillation phase aligns the pruned model's output and intermediate node and edge representations with the teacher model.

What would settle it

Run the paper's layer-wise ablation on rMD17 and at least one more dataset, removing each interaction block individually; if removing a block with higher GradCAM relevance consistently causes no larger a performance drop than removing a lower-ranked block, the ranking that selects the blocks to prune is not doing the work.

Watch

Extended reading notes

Core claim

The central claim is that structured block reduction of a pre-trained GemNet-OC backbone preserves most of JMP-L's accuracy while materially improving efficiency. Using GradCAM relevance scores, the authors rank the six interaction blocks and find diminishing returns in deeper blocks. Removing the last two blocks cuts parameters from 160M to 108M and raises inference throughput from 19.1 to 25.6 samples/s on QMOF, while the pruned model matches the full model on rMD17, SPICE, and QMOF and even outperforms it on QM9. Pre-training distillation with node-to-node and edge-to-edge losses can reduce the remaining gap, though it helps only in some settings.

Load-bearing premise

The load-bearing premise is that the gradient-based importance scores (GradCAM) correctly rank which interaction blocks are least useful, so removing the lowest-scoring blocks preserves accuracy.

Editorial extensions

If this is right

  • A 4-interaction-block JMP-L (108M parameters) can replace the 160M model on many tasks with negligible accuracy change.
  • Fine-tuning the smaller model converges as fast as or faster than the full model on tasks like QM9 within a fixed 1 GPU-day budget.
  • Resource-constrained workflows and high-throughput screening can use a 1.3x faster inference model with 32% fewer parameters.
  • Block reduction transfers to another architecture, EquiformerV2, but the gains shrink when the teacher model is already compact at 31M parameters.
  • Pre-training distillation is not uniformly beneficial downstream; it helps most for heavily pruned models and can hurt when only one or two blocks are removed.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's own layer-wise ablation suggests the GradCAM ordering is imperfect, so a simpler rule such as 'always remove the deepest blocks' might work as well without computing relevance scores.
  • Because the 4-block model beats the full model on QM9, block reduction may act as a regularizer; this could be tested by comparing fine-tuning variance across multiple random seeds.
  • The diminishing-returns pattern mirrors observations in large language models, raising the question of whether block reduction is a general property of deep pre-trained networks rather than specific to GemNet-OC.
  • A testable extension is to apply the same pipeline to other molecular backbones with publicly available pre-trained checkpoints, predicting that the benefit scales with the teacher's parameter count.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. This paper studies the layer efficiency of JMP-L, the 160M-parameter GemNet-OC-based foundation model for molecular property prediction, and proposes block reduction as a compression strategy. The authors compute GradCAM relevance scores for the concatenated per-block features on the JMP pre-training distribution, find that the later interaction blocks receive the lowest scores, and construct reduced models by removing the last interaction blocks and truncating the first FinalMLP layer accordingly (with either a 'sliced' weight initialization or a randomly re-initialized alternative). Optionally, a brief knowledge-distillation phase is run before fine-tuning, transferring output, node, and edge features from the teacher. The headline result is that removing two of six interaction blocks reduces parameters by 32% (160M to 108M) and improves inference throughput by 1.3x while incurring only a minimal accuracy drop when pruned and full models are fine-tuned under the same wall-clock budget (1 GPU-day on a V100; 2 GPU-days for QM9) and evaluated on five targets: Aspirin (rMD17), U0 (QM9), solvated amino acids (SPICE), Ac-Ala3-NHMe (MD22), and band gap (QMOF). Additional experiments cover pre-training-time MAE on the four JMP upstream datasets (with and without KD), convergence curves within the fixed budget, inference benchmarks, a layer-wise ablation on rMD17 (Appendix C), a comparison with training from scratch, and an extension to EquiformerV2 in the appendix.

Significance. If the main claim holds, the paper delivers a practically useful result: a 108M-parameter JMP variant that is 32% smaller and 1.3x faster at inference with accuracy comparable to the full model, which is valuable for high-throughput screening and resource-constrained settings. The work is one of the first layer-redundancy analyses of a large atomistic foundation model, and its conclusion is consistent with the broader observation that later transformer/GNN blocks give diminishing returns. The manuscript is transparent in several important ways: code is released; the JMP-L baseline is reproduced rather than taken from the original paper on faith; the GradCAM-based block ordering is checked against a layer-wise ablation; the EquiformerV2 study honestly reports only partial transfer; and the mixed KD results are acknowledged in Section 4.3. These strengths make the empirical core credible. The significance is nevertheless conditional: the over-parameterization conclusion rests on a fixed-wall-clock comparison that does not separate model capacity from optimization budget, on single-run results without error bars, and on an evaluation of only five of the 40 JMP tasks.

major comments (3)
  1. [§4.3, Figures 3-4; Abstract] The claim that removing two interaction blocks yields a 'minimal performance drop' is confounded by the fixed wall-clock fine-tuning budget. The authors fine-tune every configuration for 1 GPU-day on a V100 (2 GPU-days for QM9); since the 4- and 5-block models have fewer FLOPs per step, they receive proportionally more optimizer steps within that window. Equality of test MAE under this protocol therefore demonstrates efficiency per unit of wall-clock time, not that the removed blocks are superfluous at convergence. This is not a pedantic distinction: Section 4.4 (Figure 4) shows the 6-block model retaining a clear advantage on MD22 at the 1-GPU-day cutoff, and Section 2.1 states that JMP-L fine-tuning requires over 275 GPU hours to converge, so the baseline is likely far from plateau. The manuscript discloses the fixed-budget constraint ('constrained by a fixed computational budget') but does not discuss how the constraint interacts with the over-parameterization conclusion. The abstract's conclusion that 'JMP-L is over-parameterized' requires a comparison in which the 6-block model has either converged or demonstrably ceased to improve; the current protocol cannot distinguish the pruning hypothesis from the undertrained-baseline hypothesis. I ask the authors to add training curves extending past the budget cutoff (or converged fine-tuning runs) for the 6-block and 4-block models, or to reframe the central claim as fixed-budget efficiency.
  2. [Figures 3-4; Tables 1-2] All reported results are single runs with no error bars, multiple seeds, or significance tests. The differences that carry the paper's conclusion are small (e.g., in Figure 3, the 4-block and 5-block BR curves are within a few meV/Angstrom of the 6-block baseline on rMD17 and SPICE), and the KD comparisons change sign across block counts on rMD17. Without variance estimates, 'minimal performance drop' and 'comparable accuracy' are not quantitatively supported, and the reader cannot tell whether the 4-block-versus-6-block gap is real or within run-to-run noise. Please report mean and standard deviation over at least three seeds for the headline fine-tuning comparisons, and preferably for the pre-training MAE table as well.
  3. [§4.3; Abstract; §1] The downstream evaluation covers five targets (Aspirin from rMD17, U0 from QM9, solvated amino acids from SPICE, Ac-Ala3-NHMe from MD22, and band gap from QMOF) out of the 40 tasks used in [38], yet the abstract and introduction generalize to 'many tasks' and to JMP-L as such. The five targets are predominantly molecular force/energy benchmarks plus one MOF property; no OC20/OC22 adsorption or catalysis tasks are included, so the over-parameterization conclusion is not tested where JMP's pre-training capacity may matter most. Either expand the task coverage, or explicitly restrict the over-parameterization and 'comparable performance' claims to the evaluated regime.
minor comments (5)
  1. [§3.2, Appendix C] Appendix C shows an inversion in the GradCAM ordering: removing f4 hurts more than removing f3, contrary to the importance scores in Figure 2. Since the headline result removes the last two blocks (f6 and f7), this discrepancy does not invalidate the main claim, but the main text presents the scores as the justification for the pruning order without stating that the ablation validates only the global trend; a sentence acknowledging the f3/f4 caveat should appear in Section 3.2.
  2. [§1, Abstract, §4.3] The introduction's statement that 'pruning and distillation improve JMP-L's efficiency while preserving comparable performance for most tasks' and the abstract's mention of a 'pruned and distilled variant' are stronger than the results in Section 4.3: KD hurts the 4- and 5-block models on rMD17 and helps mainly at the largest reductions. The conclusion's phrasing ('knowledge distillation can help mitigate performance degradation in certain tasks') is accurate and should be used consistently in the abstract and introduction.
  3. [§3.2] The notation in the GradCAM description is imprecise: the symbol ∇CAM is never defined, and r = ReLU(f ∘ ∇CAM) should specify that the gradient is taken of L0 with respect to the concatenated feature tensor; also, the decomposition of r into per-block partitions should state the dimensions involved, since the interaction blocks and the embedding block do not share identical output shapes.
  4. [Figure 2 caption; Table 2] Figure 2's caption says 'This table illustrates...' although it is a figure, and Table 2 does not state the measurement conditions for throughput and GFlops (batch size, input graph sizes, warm-up, and whether the batch size is held constant across block counts); please specify these details.
  5. [§4.3, Figure 3] The text says the 4-block model 'outperforms the original JMP-L on the QM9 target,' but it is ambiguous whether 'original JMP-L' refers to the published converged numbers from [38], which were obtained under a different fine-tuning protocol, or to the authors' own 1-GPU-day reproduction; the comparison is only valid if the protocols are matched, so this should be clarified.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: block selection is heuristic and independently validated; the central comparison is external, though the fixed fine-tuning budget is a correctness confound, not circular.

full rationale

The paper's derivation chain is not circular. The claim that later interaction blocks contribute less is obtained from GradCAM saliency on the pre-trained JMP-L model (Section 3.2), and the pruning decision is checked by a layer-wise ablation in Appendix C, which explicitly reports a mismatch (removing f4 hurts more than f3). No parameter is fitted to downstream test MAE; the block-reduced models are fine-tuned and evaluated on held-out external benchmarks. The baseline is the reproduced JMP-L, and the original JMP-L numbers from [38] are external, with no author overlap. The only load-bearing assumption that could be challenged is the fixed 1 GPU-day fine-tuning budget (Section 4.3), which gives smaller models more optimizer steps and may understate the 6-block baseline's converged accuracy. This is an experimental confound and a threat to the over-parameterization conclusion, but it is not a circular reduction: the paper does not define the prediction in terms of the budget, nor fit a parameter renamed as a prediction. Under the stated rules, this concern belongs to correctness risk, not circularity.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The central claim relies on empirical choices (how many blocks to remove) and on several domain assumptions about the validity of GradCAM as an importance measure, the fairness of fixed compute budget comparisons, and the representativeness of the selected downstream tasks. There are no invented physical entities or fitted constants in a derivation, but the unspecified distillation loss weights and block-count choice are free parameters that affect the reported trade-offs.

free parameters (2)
  • Number of removed interaction blocks = 2
    Chosen empirically as the best accuracy/efficiency trade-off; the paper does not provide a principled criterion for this choice.
  • Knowledge distillation loss weight
    The distillation objective in Section 3.3 includes multiple loss terms, but their relative weights (including any lambda) are not reported, so this remains an unspecified hyperparameter.
assumptions (4)
  • domain assumption GradCAM relevance scores accurately reflect the contribution of interaction blocks to downstream performance.
    Section 3.2 and Appendix C; the layer-wise ablation on rMD17 shows partial agreement, but discrepancies for f3/f4 remain, so this assumption is only weakly validated.
  • domain assumption Fine-tuning under a fixed GPU-day budget is a fair comparison across model sizes.
    Section 4.3; smaller models naturally converge faster, which can favor them independent of final accuracy.
  • domain assumption The five downstream tasks (rMD17, QM9, SPICE, MD22, QMOF) are representative of JMP-L's target tasks.
    Section 4.3; only 5 of the 40 tasks from the JMP benchmark are evaluated, so generalization to the full benchmark is assumed.
  • domain assumption The JMP-L pre-trained checkpoint provides a valid starting point for pruning.
    Section 4.2; the entire method relies on the quality of this external checkpoint, which is not independently verified in this paper.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Towards Faster and More Compact Foundation Models for Molecular Property Prediction." pith.science (2026). https://pith.science/paper/I5PP6G7A

@misc{pith2026250419538,
  author       = {Pith},
  title        = {Pith review of: Towards Faster and More Compact Foundation Models for Molecular Property Prediction},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/I5PP6G7A}},
  note         = {Machine review of arXiv:2504.19538}
}
read the original abstract

Advancements in machine learning for molecular property prediction have improved accuracy but at the expense of higher computational cost and longer training times. Recently, the Joint Multi-domain Pre-training (JMP) foundation model has demonstrated strong performance across various downstream tasks with reduced training time over previous models. Despite JMP's advantages, fine-tuning it on molecular datasets ranging from small-scale to large-scale requires considerable time and computational resources. In this work, we investigate strategies to enhance efficiency by reducing model size while preserving performance. To better understand the model's efficiency, we analyze the layer contributions of JMP and find that later interaction blocks provide diminishing returns, suggesting an opportunity for model compression. We explore block reduction strategies by pruning the pre-trained model and evaluating its impact on efficiency and accuracy during fine-tuning. Our analysis reveals that removing two interaction blocks results in a minimal performance drop, reducing the model size by 32% while increasing inference throughput by 1.3x. These results suggest that JMP-L is over-parameterized and that a smaller, more efficient variant can achieve comparable performance with lower computational cost. Our study provides insights for developing lighter, faster, and more scalable foundation models for molecular and materials discovery. The code is publicly available at: https://github.com/Yasir-Ghunaim/efficient-jmp.

Figures

Figures reproduced from arXiv: 2504.19538 by the authors.

Figure 1
Figure 1. Block Reduction for Efficient Foundation Models. The top model represents the foundation model JMP-L, where interaction blocks extract features, which are concatenated and processed by FinalMLP before making predictions. The bottom model is its pruned version, constructed by removing low-importance blocks and adjusting FinalMLP. To mitigate performance degradation, we apply both feature distillation (node-to-node an… view at source ↗
Figure 2
Figure 2. Block Relevance Analysis. This table illustrates the contribution of each output block in JMP-L to the final predic￾tion. The first block, f1, represents the embedding output, while f2 to f7 correspond to the six interaction blocks of GemNet-OC. The results show diminishing returns in deeper interaction blocks, suggesting they are strong candidates for pruning to improve effi￾ciency without considerable performance … view at source ↗
Figure 3
Figure 3. Evaluation on downstream tasks. Evaluation of the performance across various downstream tasks using different block reduction strategies: block reduction (BR), block reduction with a randomly initialized MLP (BR/RandomMLP), and block reduction combined with knowledge distillation (BR+KD). Performance is measured in MAE: meV/A for force targets, meV for the QM9 energy ˚ target, and eV for the QMOF band gap target. Th… view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Training Efficiency Analysis. We compare the convergence speed of JMP-L models with 3, 4, and 6 blocks over a fixed 1 GPU-day training budget. The 4-block model achieves faster convergence than the 6-block model on QM9 after 8 hours but lags behind on MD22, while perfo…
Figure 1
Figure 1. Figure 1: Block Relevance Analysis for EquiformerV2. This figure illustrates the contribution of each transformer block in EquiformerV2 to the final prediction. The model consists of eight transformer blocks, f1 to f8, with relevance decreasing in the deeper layers. The results …
Figure 2
Figure 2. Figure 2: Evaluation on downstream tasks using EquiformerV2. We assess the performance of EquiformerV2 across various down￾stream tasks after applying block reduction (BR) and knowledge distillation (KD). Performance is reported in meV/A for force targets, ˚ meV for the QM9 ener…
Figure 3
Figure 3. Figure 3: Training from scratch vs. block reduction of a pre-trained model. Validation performance of smaller models across five downstream tasks when initialized from scratch (brown dashed line) versus when derived through block reduction from a pre-trained 6-block JMP-L model …
Figure 5
Figure 5. Figure 5: Layer-Wise Ablation Study. Validation MAE on rMD17 when individually removing each interaction block (f2 to f7) from the 6-block pre-trained GemNet-OC model. The results show that removing later blocks leads to smaller performance loss, supporting their lower importanc…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

51 extracted references · 41 canonical work pages

  1. [38]

    From molecules to materials: Pre-training large generalizable models for atomic property prediction

    Nima Shoghi, Adeesh Kolluru, John R Kitchin, Zachary W Ulissi, C Lawrence Zitnick, and Brandon M Wood. From molecules to materials: Pre-training large generalizable models for atomic property prediction. arXiv preprint arXiv:2310.16802, 2023. 1, 2, 5, 6, 8, 11

    arXiv

  2. [1]

    Ilyes Batatia, Simon Batzner, D ´avid P ´eter Kov ´acs, Albert Musaelian, Gregor N. C. Simm, Ralf Drautz, Christoph Ort- ner, Boris Kozinsky, and G´abor Cs´anyi. The design space of e(3)-equivariant atom-centered interatomic potentials, 2022. 1

    work page 2022

  3. [2]

    Ilyes Batatia, David Peter Kovacs, Gregor N. C. Simm, Christoph Ortner, and Gabor Csanyi. MACE: Higher order equivariant message passing neural networks for fast and ac- curate force fields. In Advances in Neural Information Pro- cessing Systems, 2022. 1

    work page 2022

  4. [3]

    A foundation model for atomistic materials chemistry

    Ilyes Batatia, Philipp Benner, Yuan Chiang, Alin M Elena, D´avid P Kov ´acs, Janosh Riebesell, Xavier R Advincula, Mark Asta, William J Baldwin, Noam Bernstein, et al. A foundation model for atomistic materials chemistry. arXiv preprint arXiv:2401.00096, 2023. 1, 2

    arXiv 2023

  5. [4]

    Generalized neural- network representation of high-dimensional potential-energy surfaces

    J ¨org Behler and Michele Parrinello. Generalized neural- network representation of high-dimensional potential-energy surfaces. Physical Review Letters, 98(14), 2007. 1

    work page 2007

  6. [5]

    What is the state of neural network pruning? Proceedings of machine learning and systems , 2: 129–146, 2020

    Davis Blalock, Jose Javier Gonzalez Ortiz, Jonathan Fran- kle, and John Guttag. What is the state of neural network pruning? Proceedings of machine learning and systems , 2: 129–146, 2020. 2

    work page 2020

  7. [6]

    Tucker- man, Klaus-Robert M ¨uller, and Kieron Burke

    Mihail Bogojeski, Leslie V ogt-Maranto, Mark E. Tucker- man, Klaus-Robert M ¨uller, and Kieron Burke. Quantum chemical accuracy from density functional approximations via machine learning. Nature Communications, 11(1), 2020. 1

    work page 2020

  8. [7]

    Model compression

    Cristian Bucilua, Rich Caruana, and Alexandru Niculescu- Mizil. Model compression. In Proceedings of the 12th ACM SIGKDD International Conference on Knowledge Discov- ery and Data Mining , page 535–541, New York, NY , USA,

    work page

Show all 51 references
  1. [8]

    Open catalyst 2020 (oc20) dataset and community challenges

    Lowik Chanussot, Abhishek Das, Siddharth Goyal, Thibaut Lavril, Muhammed Shuaibi, Morgane Riviere, Kevin Tran, Javier Heras-Domingo, Caleb Ho, Weihua Hu, et al. Open catalyst 2020 (oc20) dataset and community challenges. Acs Catalysis, 11(10):6059–6072, 2021. 1, 5, 8

    2020

  2. [9]

    A survey on deep neural network pruning-taxonomy, compari- son, analysis, and recommendations, 2024

    Hongrong Cheng, Miao Zhang, and Javen Qinfeng Shi. A survey on deep neural network pruning-taxonomy, compari- son, analysis, and recommendations, 2024. 2

    2024

  3. [10]

    Machine learning of accurate energy-conserving molecular force fields

    Stefan Chmiela, Alexandre Tkatchenko, Huziel E Sauceda, Igor Poltavsky, Kristof T Sch ¨utt, and Klaus-Robert M ¨uller. Machine learning of accurate energy-conserving molecular force fields. Science advances, 3(5):e1603015, 2017. 6

    2017

  4. [11]

    Accurate global machine learning force fields for molecules with hundreds of atoms

    Stefan Chmiela, Valentin Vassilev-Galindo, Oliver T Unke, Adil Kabylda, Huziel E Sauceda, Alexandre Tkatchenko, and Klaus-Robert M¨uller. Accurate global machine learning force fields for molecules with hundreds of atoms. Science Advances, 9(2):eadf0873, 2023. 6

    2023

  5. [12]

    Imagenet: A large-scale hierarchical image database

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In IEEE Conference on Computer Vision and Pat- tern Recognition, pages 248–255, 2009. 2

    2009

  6. [13]

    An image is worth 16x16 words: Transformers for image recognition at scale

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Syl- vain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition ...

    2021

  7. [14]

    Benchmarking materials property prediction methods: the matbench test set and automatminer reference algorithm

    Alexander Dunn, Qi Wang, Alex Ganose, Daniel Dopp, and Anubhav Jain. Benchmarking materials property prediction methods: the matbench test set and automatminer reference algorithm. npj Computational Materials, 6(1):138, 2020. 1

    2020

  8. [15]

    Spice, a dataset of drug-like molecules and peptides for training machine learning potentials

    Peter Eastman, Pavan Kumar Behara, David L Dotson, Rai- mondas Galvelis, John E Herr, Josh T Horton, Yuezhi Mao, John D Chodera, Benjamin P Pritchard, Yuanqing Wang, et al. Spice, a dataset of drug-like molecules and peptides for training machine learning potentials. Scientifi...

    2023

  9. [16]

    Accelerating molecular graph neu- ral networks via knowledge distillation

    Filip Ekstr ¨om Kelvinius, Dimitar Georgiev, Artur Toshev, and Johannes Gasteiger. Accelerating molecular graph neu- ral networks via knowledge distillation. Advances in Neural Information Processing Systems, 36, 2024. 2, 3, 5

    2024

  10. [17]

    Gemnet: Universal directional graph neural networks for molecules

    Johannes Gasteiger, Florian Becker, and Stephan G¨unnemann. Gemnet: Universal directional graph neural networks for molecules. Advances in Neural Information Processing Systems, 34:6790–6802, 2021. 3

    2021

  11. [18]

    Gemnet-oc: developing graph neural net- works for large and diverse molecular simulation datasets

    Johannes Gasteiger, Muhammed Shuaibi, Anuroop Sriram, Stephan G ¨unnemann, Zachary Ulissi, C Lawrence Zitnick, and Abhishek Das. Gemnet-oc: developing graph neural net- works for large and diverse molecular simulation datasets. arXiv preprint arXiv:2204.02782, 2022. 2, 3, 8

    2022 arXiv

  12. [19]

    Andrey Gromov, Kushal Tirumala, Hassan Shapourian, Paolo Glorioso, and Daniel A. Roberts. The unreasonable ineffectiveness of the deeper layers, 2024. 2, 3

    2024

  13. [20]

    Knowl- edge distillation in vision transformers: A critical review,

    Gousia Habib, Tausifa Jan Saleem, and Brejesh Lall. Knowl- edge distillation in vision transformers: A critical review,

  14. [21]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceed- ings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016. 2

    2016

  15. [22]

    Distilling the knowledge in a neural net- work

    Geoffrey Hinton. Distilling the knowledge in a neural net- work. arXiv preprint arXiv:1503.02531, 2015. 2, 3

    2015 arXiv

  16. [23]

    Lawrence Zit- nick, John R Kitchin, and Zachary W Ulissi

    Adeesh Kolluru, Muhammed Shuaibi, Aini Palizhati, Nima Shoghi, Abhishek Das, Brandon Wood, C. Lawrence Zit- nick, John R Kitchin, and Zachary W Ulissi. Open chal- lenges in developing generalizable large scale machine learn- ing models for catalyst discovery, 2022. 1

    2022

  17. [24]

    Mace- off23: Transferable machine learning force fields for organic molecules

    D ´avid P ´eter Kov ´acs, J Harry Moore, Nicholas J Brown- ing, Ilyes Batatia, Joshua T Horton, Venkat Kapil, Ioan- Bogdan Magd˘au, Daniel J Cole, and G ´abor Cs´anyi. Mace- off23: Transferable machine learning force fields for organic molecules. arXiv preprint arXiv:2312.1521...

    2023 arXiv

  18. [25]

    Equiformerv2: Improved equivariant transformer for scaling to higher-degree representations

    Yi-Lun Liao, Brandon Wood, Abhishek Das, and Tess Smidt. Equiformerv2: Improved equivariant transformer for scaling to higher-degree representations. arXiv preprint arXiv:2306.12059, 2023. 8, 11

    2023 arXiv

  19. [26]

    Comprehen- sive graph gradual pruning for sparse training in graph neural networks, 2022

    Chuang Liu, Xueqi Ma, Yibing Zhan, Liang Ding, Dapeng Tao, Bo Du, Wenbin Hu, and Danilo Mandic. Comprehen- sive graph gradual pruning for sparse training in graph neural networks, 2022. 2

    2022

  20. [27]

    LLM- pruner: On the structural pruning of large language mod- els

    Xinyin Ma, Gongfan Fang, and Xinchao Wang. LLM- pruner: On the structural pruning of large language mod- els. In Thirty-seventh Conference on Neural Information Processing Systems, 2023. 3

    2023

  21. [28]

    Nørskov, Frank Abild-Pedersen, Felix Studt, and Thomas Bligaard

    Jens K. Nørskov, Frank Abild-Pedersen, Felix Studt, and Thomas Bligaard. Density functional theory in surface chemistry and catalysis. Proceedings of the National Academy of Sciences, 108(3):937–943, 2011. 1

    2011

  22. [29]

    Quantum chemistry structures and properties of 134 kilo molecules

    Raghunathan Ramakrishnan, Pavlo O Dral, Matthias Rupp, and O Anatole V on Lilienfeld. Quantum chemistry structures and properties of 134 kilo molecules. Scientific data, 1(1): 1–7, 2014. 6

    2014

  23. [30]

    Machine learning the quantum-chemical properties of metal–organic frameworks for accelerated materials discovery.Matter, 4(5):1578–1597,

    Andrew S Rosen, Shaelyn M Iyer, Debmalya Ray, Zhen- peng Yao, Alan Aspuru-Guzik, Laura Gagliardi, Justin M Notestein, and Randall Q Snurr. Machine learning the quantum-chemical properties of metal–organic frameworks for accelerated materials discovery.Matter, 4(5):1578–1597,

  24. [31]

    Rosen, Victor Fung, Patrick Huck, Cody T

    Andrew S. Rosen, Victor Fung, Patrick Huck, Cody T. O’Donnell, Matthew K. Horton, Donald G. Truhlar, Kristin A. Persson, Justin M. Notestein, and Randall Q. Snurr. High-throughput predictions of metal–organic frame- work electronic properties: theoretical challenges, graph neu...

    2022

  25. [32]

    Sabe, Thandokuhle Ntombela, Lindiwe A

    Victor T. Sabe, Thandokuhle Ntombela, Lindiwe A. Jhamba, Glenn E.M. Maguire, Thavendran Govender, Tricia Naicker, and Hendrik G. Kruger. Current trends in computer aided drug design and a highlight of drugs discovered via computa- tional techniques: A review. European Journal ...

    2021

  26. [33]

    Transition1x - a dataset for building generalizable reactive machine learning potentials

    Mathias Schreiner, Arghya Bhowmik, Tejs Vegge, Jonas Busk, and Ole Winther. Transition1x - a dataset for building generalizable reactive machine learning potentials. Scientific Data, 9(1), 2022. 1

    2022

  27. [34]

    Transition1x-a dataset for building generalizable reactive machine learning potentials

    Mathias Schreiner, Arghya Bhowmik, Tejs Vegge, Jonas Busk, and Ole Winther. Transition1x-a dataset for building generalizable reactive machine learning potentials. Scientific Data, 9(1):779, 2022. 5

    2022

  28. [35]

    Schnet: A continuous-filter con- volutional neural network for modeling quantum interac- tions

    Kristof Sch ¨utt, Pieter-Jan Kindermans, Huziel Enoc Sauceda Felix, Stefan Chmiela, Alexandre Tkatchenko, and Klaus-Robert M ¨uller. Schnet: A continuous-filter con- volutional neural network for modeling quantum interac- tions. Advances in neural information processing system...

    2017

  29. [36]

    Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Ba- tra

    Ramprasaath R. Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Ba- tra. Grad-cam: Visual explanations from deep networks via gradient-based localization. In 2017 IEEE International Conference on Computer Vision (ICCV) , pages 618–626,

    2017

  30. [37]

    Lorenzini, Rui Ma, Qiang Zhu, Daniel W

    Vinit Sharma, Chenchen Wang, Robert G. Lorenzini, Rui Ma, Qiang Zhu, Daniel W. Sinkovits, Ghanshyam Pila- nia, Artem R. Oganov, Sanat Kumar, Gregory A. Sotzing, Steven A. Boggs, and Rampi Ramprasad. Rational design of all organic polymer dielectrics. Nature Communications, 5 (...

    2014

  31. [39]

    Neural net pruning-why and how

    Sietsma and Dow. Neural net pruning-why and how. InIEEE 1988 International Conference on Neural Networks , pages 325–333 vol.1, 1988. 2

    1988

  32. [40]

    The ani-1ccx and ani- 1x data sets, coupled-cluster and density functional theory properties for molecules

    Justin S Smith, Roman Zubatyuk, Benjamin Nebgen, Nicholas Lubbers, Kipton Barros, Adrian E Roitberg, Olexandr Isayev, and Sergei Tretiak. The ani-1ccx and ani- 1x data sets, coupled-cluster and density functional theory properties for molecules. Scientific data, 7(1):134, 2020. 1, 5

    2020

  33. [41]

    Brab- son, Abhishek Das, Zachary Ulissi, Matt Uyttendaele, An- drew J

    Anuroop Sriram, Sihoon Choi, Xiaohan Yu, Logan M. Brab- son, Abhishek Das, Zachary Ulissi, Matt Uyttendaele, An- drew J. Medford, and David S. Sholl. The open dac 2023 dataset and challenges for sorbent discovery in direct air cap- ture, 2023. 1

    2023

  34. [42]

    Zico Kolter

    Mingjie Sun, Zhuang Liu, Anna Bair, and J. Zico Kolter. A simple and effective pruning approach for large language models, 2024. 2

    2024

  35. [43]

    Yijun Tian, Shichao Pei, Xiangliang Zhang, Chuxu Zhang, and Nitesh V . Chawla. Knowledge distillation on graphs: A survey, 2023. 2

    2023

  36. [44]

    The open catalyst 2022 (oc22) dataset and challenges for oxide electrocatalysts

    Richard Tran, Janice Lan, Muhammed Shuaibi, Bran- don M Wood, Siddharth Goyal, Abhishek Das, Javier Heras- Domingo, Adeesh Kolluru, Ammar Rizvi, Nima Shoghi, et al. The open catalyst 2022 (oc22) dataset and challenges for oxide electrocatalysts. ACS Catalysis, 13(5):3066–3084,

    2022

  37. [45]

    A survey on knowledge distillation of large language models,

    Xiaohan Xu, Ming Li, Chongyang Tao, Tao Shen, Reynold Cheng, Jinyang Li, Can Xu, Dacheng Tao, and Tianyi Zhou. A survey on knowledge distillation of large language models,

  38. [46]

    Width & depth pruning for vision transformers

    Fang Yu, Kun Huang, Meng Wang, Yuan Cheng, Wei Chu, and Li Cui. Width & depth pruning for vision transformers. In Proceedings of the AAAI Conference on Artificial Intelli- gence, pages 3143–3151, 2022. 2

    2022

  39. [47]

    Pre- training via denoising for molecular property prediction,

    Sheheryar Zaidi, Michael Schaarschmidt, James Martens, Hyunjik Kim, Yee Whye Teh, Alvaro Sanchez-Gonzalez, Pe- ter Battaglia, Razvan Pascanu, and Jonathan Godwin. Pre- training via denoising for molecular property prediction,

  40. [48]

    Molkd: Distilling cross-modal knowledge in chemical reactions for molecular property prediction, 2023

    Liang Zeng, Lanqing Li, and Jian Li. Molkd: Distilling cross-modal knowledge in chemical reactions for molecular property prediction, 2023. 2

    2023

  41. [49]

    Loraprune: Structured pruning meets low-rank parameter-efficient fine- tuning, 2024

    Mingyang Zhang, Hao Chen, Chunhua Shen, Zhen Yang, Linlin Ou, Xinyi Yu, and Bohan Zhuang. Loraprune: Structured pruning meets low-rank parameter-efficient fine- tuning, 2024. 2

    2024

  42. [50]

    Uni-mol: A universal 3d molecular representation learning framework

    Gengmo Zhou, Zhifeng Gao, Qiankun Ding, Hang Zheng, Hongteng Xu, Zhewei Wei, Linfeng Zhang, and Guolin Ke. Uni-mol: A universal 3d molecular representation learning framework. In The Eleventh International Conference on Learning Representations, 2023. 1 Appendix A. Beyond GemN...

    2023

  43. [2006]

    Association for Computing Machinery. 2

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.