MolGram integrates a conditional n-gram memory module into molecular language models to address locality gaps in SMILES tokenization, improving performance on generation, forward prediction, and retrosynthesis while outperforming 3x larger baselines.
SMILES Enumeration as Data Augmentation for Neural Network Modeling of Molecules
8 Pith papers cite this work, alongside 309 external citations. Polarity classification is still indexing.
8
Pith papers citing it
309
external citations · Pith
abstract
Simplified Molecular Input Line Entry System (SMILES) is a single line text representation of a unique molecule. One molecule can however have multiple SMILES strings, which is a reason that canonical SMILES have been defined, which ensures a one to one correspondence between SMILES string and molecule. Here the fact that multiple SMILES represent the same molecule is explored as a technique for data augmentation of a molecular QSAR dataset modeled by a long short term memory (LSTM) cell based neural network. The augmented dataset was 130 times bigger than the original. The network trained with the augmented dataset shows better performance on a test set when compared to a model built with only one canonical SMILES string per molecule. The correlation coefficient R2 on the test set was improved from 0.56 to 0.66 when using SMILES enumeration, and the root mean square error (RMS) likewise fell from 0.62 to 0.55. The technique also works in the prediction phase. By taking the average per molecule of the predictions for the enumerated SMILES a further improvement to a correlation coefficient of 0.68 and a RMS of 0.52 was found.
citation-role summary
background 2
citation-polarity summary
verdicts
UNVERDICTED 8roles
background 2polarities
background 2representative citing papers
Chirality emerges in SMILES translation models through an abrupt encoder-centered reorganization of representations after a long plateau, identified via checkpoint analysis and ablation.
Canonization produces generalization bounds ranging from invariant-optimal to non-invariant depending on regularity, with Hilbert-curve ordering proven to give polynomial covering-number growth for point clouds while lexicographic sorting gives exponential growth.
Sparse autoencoders on MolFormer reveal position-tracking latents in early layers and atom-in-substructure plus pharmacologically relevant features in later layers, with non-canonical SMILES causing greater representation disruption than invalid ones.
Chemically meaningful steering for properties like cLogP and TPSA emerges in entangled Transformer-VAE latent spaces only after controlling for SELFIES representation confounds through residualization and decoded traversals.
A multimodal deep learning approach using heterogeneous representations and network types achieves significantly higher accuracy than state-of-the-art methods on a standard toxicity benchmark.
A 53K-parameter weight-shared transformer generates novel valid SMILES at 95% rate on ZINC-250K and resolves constraints hierarchically via bracket, ring, and valence stages as shown by probing and ablation.
GLACIER combines graph, SMILES, and descriptor encoders with Finsler fusion and contrastive distillation to produce an efficient multimodal model for molecular property prediction.
citing papers explorer
-
Augmenting Molecular Language Models with Local $n$-gram Memory
MolGram integrates a conditional n-gram memory module into molecular language models to address locality gaps in SMILES tokenization, improving performance on generation, forward prediction, and retrosynthesis while outperforming 3x larger baselines.
-
From Syntax to Semantics: Unveiling the Emergence of Chirality in SMILES Translation Models
Chirality emerges in SMILES translation models through an abrupt encoder-centered reorganization of representations after a long plateau, identified via checkpoint analysis and ablation.
-
When and How to Canonize: A Generalization Perspective
Canonization produces generalization bounds ranging from invariant-optimal to non-invariant depending on regularity, with Hilbert-curve ordering proven to give polynomial covering-number growth for point clouds while lexicographic sorting gives exponential growth.
-
What Does a Chemical Language Model Know About Molecules?
Sparse autoencoders on MolFormer reveal position-tracking latents in early layers and atom-in-substructure plus pharmacologically relevant features in later layers, with non-canonical SMILES causing greater representation disruption than invalid ones.
-
Molecules Meet Language: Confound-Aware Representation Learning and Chemical Property Steering in Transformer-VAE Latent Spaces
Chemically meaningful steering for properties like cLogP and TPSA emerges in entangled Transformer-VAE latent spaces only after controlling for SELFIES representation confounds through residualization and decoded traversals.
-
Toxicity Prediction by Multimodal Deep Learning
A multimodal deep learning approach using heterogeneous representations and network types achieves significantly higher accuracy than state-of-the-art methods on a standard toxicity benchmark.
-
SMolLM: Small Language Models Learn Small Molecular Grammar
A 53K-parameter weight-shared transformer generates novel valid SMILES at 95% rate on ZINC-250K and resolves constraints hierarchically via bracket, ring, and valence stages as shown by probing and ablation.
-
GLACIER: A Multimodal Student-Teacher Foundation Model for Molecular Property Prediction
GLACIER combines graph, SMILES, and descriptor encoders with Finsler fusion and contrastive distillation to produce an efficient multimodal model for molecular property prediction.