Pith. sign in

REVIEW 5 cited by

Towards 3D Molecule-Text Interpretation in Language Models

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2401.13923 v2 pith:KUC7UFUF submitted 2024-01-25 cs.LG cs.IRq-bio.BM

Sihang Li , Zhiyuan Liu , Yanchen Luo , Xiang Wang , Xiangnan He , Kenji Kawaguchi , Tat-Seng Chua , Qi Tian This is my paper
classification cs.LGcs.IRq-bio.BM
keywords moleculard-molmmolecule-textencoderinstructionlanguageintegrationinterpretation
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Language Models (LMs) have greatly influenced diverse domains. However, their inherent limitation in comprehending 3D molecular structures has considerably constrained their potential in the biomolecular domain. To bridge this gap, we focus on 3D molecule-text interpretation, and propose 3D-MoLM: 3D-Molecular Language Modeling. Specifically, 3D-MoLM enables an LM to interpret and analyze 3D molecules by equipping the LM with a 3D molecular encoder. This integration is achieved by a 3D molecule-text projector, bridging the 3D molecular encoder's representation space and the LM's input space. Moreover, to enhance 3D-MoLM's ability of cross-modal molecular understanding and instruction following, we meticulously curated a 3D molecule-centric instruction tuning dataset -- 3D-MoIT. Through 3D molecule-text alignment and 3D molecule-centric instruction tuning, 3D-MoLM establishes an integration of 3D molecular encoder and LM. It significantly surpasses existing baselines on downstream tasks, including molecule-text retrieval, molecule captioning, and more challenging open-text molecular QA tasks, especially focusing on 3D-dependent properties. We release our codes and datasets at https://github.com/lsh0520/3D-MoLM.

Discussion (0). Sign in to comment.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Atomistic Language Models Understand and Generate Materials

    cs.LG 2026-06 unverdicted novelty 7.0 of 10

    ALMs unify pretrained atomistic encoder, LLM, and denoising diffusion via continuous projectors and staged training to reach SOTA on text-conditioned crystal prediction and de novo generation.

  2. Hypothesis-and-Refinement Learning of Organic Structures from Multimodal Spectroscopic Data

    physics.chem-ph 2026-07 conditional novelty 6.0 of 10

    A two-stage AI pipeline — spectral hypothesis generation followed by mass-constrained molecular refinement — reconstructs organic structures from multimodal spectra, with 93.8% top-1 accuracy on simulated QM9 data and...

  3. MolDA: Molecular Understanding and Generation via Large Language Diffusion Model

    cs.AI 2026-04 unverdicted novelty 6.0 of 10

    MolDA is a multimodal molecular model that uses a discrete large language diffusion backbone plus a hybrid graph encoder to achieve better global coherence and validity than autoregressive approaches.

  4. MolReFlect: Towards In-Context Fine-grained Alignments between Molecules and Texts

    cs.CL 2024-11 unverdicted novelty 6.0 of 10

    MolReFlect introduces a teacher-student framework that automatically creates fine-grained molecule-text alignments to achieve SOTA results on molecule-caption translation.

  5. SciCore-Mol: Augmenting Large Language Models with Pluggable Molecular Cognition Modules

    cs.AI 2026-05 unverdicted novelty 5.0 of 10

    SciCore-Mol augments LLMs with three integrated modules for molecular perception, latent diffusion generation, and reaction reasoning, claiming an 8B open model competes with or exceeds proprietary systems on chemical tasks.

Pith tools