{"id":"9cfb10ea-f2e3-4569-b919-5ec8efe2c924","arxiv_id":"2412.01564","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"Mol-StrucTok tokenizes 3D molecular coordinates via a spherical line notation and VQ-VAE, enabling fast GPT-2 based generation and small property-prediction improvements.","lead":"This paper introduces Mol-StrucTok, a method that turns 3D molecular structures into discrete token sequences using local spherical coordinates and a learned codebook. It reports much faster molecule generation than diffusion models with comparable stability, plus modest gains in property prediction.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The local frame construction in Appendix A (Eqs. 17-18) divides by a zero vector when the three reference atoms are collinear, so the spherical descriptor is undefined for linear fragments such as terminal alkynes; the central tokenization thus has a hidden domain restriction.","rationale":"My independent read lands on the same load-bearing assumption identified by the reader: the spherical coordinate frame is only valid when the three reference atoms are non-collinear, and the paper does not address the degenerate case. This is the most central issue because the entire tokenization pipeline, from VQ-VAE descriptors to GPT-2 vocabulary, presupposes a well-defined (d, θ, φ) for every atom. The conditional-generation results close to oracle and the low uniqueness are worth further scrutiny, but those are evaluation-quality concerns; the collinear-reference problem is an internal correctness gap in the core construction. I agree with the reader's conditional verdict: the concern is concrete and testable, but it does not necessarily sink the method if the implementation contains a sound fallback or if the incidence in QM9/PCQM4Mv2 is negligible. The machine-checkable way to settle it is to instrument the released code and count degeneracies on the actual training and evaluation data, which is why I recommend keeping the verdict unchanged pending that check.","tokens_in":16289,"tokens_out":8943,"duration_ms":86109,"concrete_test":"Add an assertion to the released descriptor-extraction code that flags any atom with ||x_c2f - (x_c2f·e1)e1|| < 1e-8 (or NaN after normalization) and run it on the PCQM4Mv2 training split and the QM9 split used in Tables 1-3; separately encode acetylene, 1,3-butadiyne, allene, and any linear di-substituted alkyne. If the flagged count is nonzero, or any of these small linear molecules yields an undefined e2, the method as described requires a fallback and the reported validity and conditional-generation numbers need to be re-examined on the non-degenerate subset.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 3.2 defines the 2D topology-based reference rule (Eq. 5): for atom vi, f is the most recent previous topological neighbor, c1 = F(f), and c2 = F(c1). Appendix A then builds an orthonormal frame via Gram-Schmidt: e1 = x_c1f/||x_c1f|| and e2 = (x_c2f - (x_c2f·e1)e1)/||...||. If c2, c1, and f are collinear, x_c2f is parallel to x_c1f, the numerator of e2 is exactly zero, and e2 is undefined. This occurs for any linear triatomic fragment used as the reference chain, e.g., a terminal alkyne R-C≡C-R' when placing an atom on the far carbon, or an allene heavy-atom chain. The paper does not mention this degeneracy or provide a fallback rule. Because the descriptor zi (Eq. 9) and all downstream tokens depend on θi and φi measured from this frame, the tokenization is not well-defined on a nonempty class of valid molecules. This is a correctness issue internal to the central construction, not a matter of disagreement with the field: the released implementation may silently skip, perturb, or otherwise special-case such atoms, in which case the reported numbers come from a procedure different from the one described. The claim that Mol-StrucTok provides a general 3D line notation therefore requires either a demonstrated absence of such cases or an explicit degeneracy-handling strategy.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Mol-StrucTok, a method that converts 3D molecular conformations into discrete tokens by appending local spherical coordinates to an existing 2D line notation (SMILES or SELFIES) and then applying VQ-VAE quantization. The resulting tokens are used as a structure-aware vocabulary for a GPT-2 style autoregressive generator, and as additional embeddings for Graphormer on QM9 property prediction. The authors report competitive validity and stability, a roughly 28x sampling speedup over diffusion baselines, and large improvements in conditional generation MAE over GeoBFN.","tokens_in":16619,"tokens_out":5120,"duration_ms":46898,"significance":"If the construction is made fully well-defined, the paper would be a useful contribution: it provides a way to adapt language-model machinery to 3D molecular structure, with a concrete tokenization scheme and a separation between tokenizer training on PCQM4Mv2 and downstream evaluation on QM9, which avoids obvious circularity. The robustness comparison of three reference-frame choices in Section 5 is a nice empirical sanity check. However, the central tokenization has correctness gaps (degenerate local frames and atoms with fewer than four neighbors), and the headline conditional-generation results need additional scrutiny before the claims can be accepted.","major_comments":[{"comment":"The local frame is undefined whenever the reference atoms v_f, v_c1, and v_c2 are collinear, because the Gram-Schmidt numerator x_c2f - (x_c2f·e1)e1 becomes exactly zero and e2 is undefined. With the 2D topology-based rule of Eq. (5), this occurs for linear fragments such as terminal alkynes, allene heavy-atom chains, and CO2. The paper neither mentions this degeneracy nor provides a fallback rule (e.g., skipping to the next non-collinear topological neighbor or using a fixed perpendicular vector). Since the descriptor z_i in Eq. (9) and every downstream token depend on this frame, the tokenization is not well-defined on a nonempty class of valid molecules. The authors must specify a degeneracy-handling strategy and either prove such cases do not occur in their datasets or report how they are processed in the released implementation.","section":"Section 3.2 and Appendix A, Eqs. (17)-(18)"},{"comment":"The understanding descriptor u_i assumes that every atom has at least four nearest neighbors, but molecules in QM9 and PCQM4Mv2 contain terminal atoms (including hydrogen) and linear molecules such as CO2 whose central atom has only two neighbors. The paper does not describe how the missing bond lengths and bond angles are encoded (padding, sentinel values, or exclusion). This is a load-bearing domain restriction for the VQ-VAE input, and the reported reconstruction and generation numbers cannot be interpreted without knowing how these cases are handled.","section":"Section 3.3, Eq. (8)"},{"comment":"The conditional-generation MAEs reported for Mol-StrucTok approach the oracle row (e.g., 0.33 vs 0.10 for alpha, 89 vs 64 meV for Delta-epsilon), while unconditional uniqueness is substantially lower than baselines in Table 1 (85.35% vs 98.85% for GeoLDM). This pattern is consistent with the generator reproducing near-training-set molecules rather than performing genuine condition-controlled sampling. To support the claim of 'significant improvements over GeoBFN', the paper should report variance over seeds, the distance of generated molecules to the training set (e.g., nearest-neighbor in the descriptor space), and the diversity of condition-matched molecules.","section":"Section 4.3, Table 3"},{"comment":"The reported gains from adding Mol-StrucTok embeddings to Graphormer are small and are presented without error bars or significance tests: epsilon_HOMO 46 to 42 meV, epsilon_LUMO 47 to 39 meV, Delta-epsilon 66 to 62 meV. Given that the base Graphormer numbers are only single runs, these differences may be within run-to-run noise. Paired runs with variance estimates are needed before claiming 'consistent improvements'.","section":"Section 4.4, Table 4"}],"minor_comments":[{"comment":"The notation N(i,j) is used in the argmax as though it were an indicator of topological adjacency, but it is defined only as 'topological neighbors of atom vi with respect to atom vj'; please replace it with an explicit indicator function or a clear definition.","section":"Section 3.2, Eq. (5)"},{"comment":"The text says the two reference atoms are the closest to the focal atom, but the formula uses argmax over Euclidean distance, which selects the farthest atoms. This is contradictory and should be corrected to argmin if the text is meant literally.","section":"Section 3.2, Eq. (6)"},{"comment":"The descriptor z_i is said to lie in R^14, but it includes sign(phi_i), which is a binary categorical variable; the encoding of this sign (as +1/-1, 0/1, or a one-hot vector) should be stated explicitly.","section":"Section 3.3, Eq. (9)"},{"comment":"The rotation-invariance derivation contains a confusing chain in Eq. (33) where primed and unprimed vectors are mixed; additionally, for phi_i, equality of cosines alone does not determine the angle on (-pi, pi], so the proof should state explicitly that proper rotations preserve the orientation of the frame and that the sign component of the descriptor is handled consistently.","section":"Appendix B.2, Eqs. (33) and (38)"},{"comment":"The sentence 'We applied a repetition penalty of 1' is ambiguous, since a repetition penalty of 1 typically means no penalty; please clarify the value and its effect.","section":"Appendix C.2"},{"comment":"The method name is typeset inconsistently as 'MOL-STRUCTOK' in Table 2 while the rest of the paper uses 'Mol-StrucTok'.","section":"Table 2"}],"recommendation":"major_revision","confidential_remarks":"The stress-test concern about collinear reference atoms lands: it is a genuine gap in the central construction, and the missing fewer-than-four-neighbors handling compounds it. Both are fixable within the manuscript's scope, so I do not recommend rejection, but the authors should be asked to address them explicitly and to report the frequency of degenerate cases in their datasets. The conditional-generation results, if they survive the memorization and variance checks, would be the strongest claim in the paper; at present they are not yet convincing."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Mol-StrucTok is a genuinely new idea for 3D molecule generation with language models: wrap an existing 2D line notation, attach local spherical coordinates computed without a learned model, quantize the descriptors with a VQ-VAE trained on PCQM4Mv2, and generate with GPT-2. The speed result is striking — 39.8 samples per second on an A100 versus about 1.4 for EDM/GeoLDM — and the unconditional validity and PoseBusters rates are competitive. The paper also shows the learned tokens help a 2D Graphormer close part of the gap to 3D models on QM9 property prediction. The comparison of three reference-atom selection rules, with a noise-based robustness test, is a nice touch.\n\nThe main soft spot is a correctness gap in the coordinate frame construction. Appendix A builds the orthonormal frame via Gram-Schmidt on the vectors from the focal atom to c1 and c2. If those two reference atoms and the focal atom are collinear, the numerator of e2 is identically zero and the descriptor is undefined. That happens for linear fragments — a terminal alkyne or an allene heavy-atom chain — which are present in QM9 and PCQM4Mv2. The paper never mentions this degeneracy or a fallback rule. The released code may silently skip or perturb such atoms, which would mean the reported numbers come from a different procedure than the one described. This is checkable and fixable, but it is a real gap in a load-bearing part of the method.\n\nTwo other issues are worth naming but less severe. Tables 3 and 4 have no error bars or seed variance, which matters because the conditional MAEs are close to the oracle values (0.33 vs 0.10 for α; 89 vs 64 meV for Δε). The authors attribute this to LM determinism; that is plausible, but a reviewer should ask for repeated runs and a sanity check that the EGNN classifier isn't being exploited. Uniqueness is also clearly lower than the diffusion baselines (around 85% versus 98-99%), so the diversity trade-off deserves a more careful analysis than one temperature curve.\n\nThe SE(3) invariance proof is basically right, though the write-up has typos and one or two confusing lines. The lack of an empirical comparison to other LM-based 3D tokenization methods (BindGPT, the geometry-informed tokenization cited in the related work) is a missed benchmark rather than a flaw.\n\nThis paper is worth a serious referee. I would send it out, ask for the degeneracy to be handled explicitly, and request error bars and a diversity analysis. If those come back clean, it's a solid contribution to molecular generation.","headline":"Mol-StrucTok is a fresh, practical 3D tokenization method with a real speed advantage, but the unhandled collinearity degeneracy and missing error bars require revision before I'd trust the headline numbers.","tokens_in":17178,"tokens_out":3998,"would_cite":true,"duration_ms":32101,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Treating each atom's local position as a discrete spherical-coordinate token turns 3D molecule generation into a language-model task and runs about 28 times faster than diffusion.","keywords":["3D molecule generation","tokenization","spherical coordinates","vector quantization","language models","SE(3) invariance","QM9 property prediction","SELFIES"],"falsifier":"Run the tokenizer's descriptor extraction on a linear triatomic molecule such as CO₂: at atoms where the focal atom and its two reference atoms are collinear, the Gram-Schmidt denominator in the construction of $e_2$ becomes exactly zero, so no valid $(d_i, \\theta_i, \\varphi_i)$ descriptor exists and the pipeline either crashes, assigns an arbitrary frame, or fails to reconstruct the molecule.","tokens_in":16086,"feed_emoji":"🧪","tokens_out":9686,"duration_ms":79152,"temperature":0.7,"pith_summary":"The paper sets out to make 3D molecular structure as natural for language models as SMILES strings are for 2D molecules, and claims this can be done by quantizing each atom's position into discrete tokens rather than predicting continuous coordinates. It introduces a spherical line notation: every atom token carries a local distance $d_i$, polar angle $\\theta_i$, and azimuth $\\varphi_i$ measured in a frame built from a focal atom and two reference atoms, which makes the coordinates invariant under rotation and translation. A VQ-VAE turns these continuous coordinates, together with local bond lengths and angles, into one of 256 codebook tokens, and a GPT-2 style model is trained to generate the resulting token sequence. The paper reports that the model produces valid, chemically stable molecules, matches diffusion methods in quality, generates them about 28 times faster, and conditions more accurately on quantum properties.","feed_headline":"Molecule shapes as text tokens generate 3D structures 28x faster","feed_subtitle":"Atomic positions become discrete spherical tokens, letting language models output valid 3D molecules.","key_machinery":"The load-bearing object is the per-atom spherical descriptor $z_i = (d_i, \\theta_i, |\\varphi_i|, \\mathrm{sign}(\\varphi_i), l_{j_1}, l_{j_2}, l_{j_3}, l_{j_4}, \\alpha_{j_1 i j_2}, \\ldots)$, a 14-dimensional vector that is invariant under rotations and translations by construction. It carries the argument in two ways. First, it linearizes a 3D structure: when appended to the atom tokens of SMILES or SELFIES, the sequence of descriptors determines the full geometry. Second, the VQ-VAE maps it to one of 256 codebook entries, giving the discrete structural alphabet that a GPT-2 model can predict autoregressively. The reference-frame construction uses topology-based selection, where the focal atom is the closest already-written neighbor in the molecular graph and its references are that neighbor's predecessors, which the paper shows is more robust to coordinate noise than 1D sequence-based or 3D distance-based selection.","core_discovery":"The central discovery is that an SE(3)-invariant discretization of local geometry is sufficient to reduce 3D molecule generation to next-token prediction without sacrificing chemical validity. For each atom, Mol-StrucTok appends to its ordinary line-notation token a 14-dimensional descriptor: the distance $d_i$, polar angle $\\theta_i$, azimuth $\\varphi_i$, and its sign, measured in a local frame fixed by a focal atom and two reference atoms chosen by 2D topology, followed by the bond lengths and bond angles to the atom's four nearest neighbors. A VQ-VAE with a 256-entry codebook quantizes that descriptor into a single discrete token, so the vocabulary is the product of atom types and structural alphabet entries. Trained as a GPT-2 next-token predictor on QM9, the paper claims 98.02% validity by lookup table, 88.30% molecular stability, and a 39.8 samples per second generation throughput versus 1.4 for the diffusion baselines, a roughly 28 times speedup; in conditional generation it reports a gap-energy MAE of 89 meV, where the prior best baseline GeoBFN reports 577 meV.","pith_inferences":["The paper leaves implicit that its discrete interface should accept any text-style conditioning, not just a scalar property; a testable extension is to prepend natural-language or protein-pocket constraints and fine-tune the same GPT-2 model.","The 256-token codebook is a likely diversity bottleneck, since uniqueness (about 85%) trails diffusion baselines; enlarging the codebook or adding stochastic decoding is an untested way to close that gap.","No degenerate-case handling is given for collinear reference atoms; until a fallback frame is defined, linear molecules are a boundary case the tokenizer likely cannot represent.","Because the tokenizer is SE(3)-invariant and line-notation agnostic, it should transfer to conformer generation or to other small-molecule datasets; the paper only demonstrates QM9, so that generality is an inference."],"forward_implications":["Molecule generation can ride on the same infrastructure as large language models, including hardware-accelerated autoregressive decoding and KV-cache, which is why the reported throughput reaches about 40 samples per second on one A100 GPU.","Property-conditioned generation becomes a conditional language-model problem; on QM9 the method reports a gap-energy MAE of 89 meV against 577 meV for GeoBFN, suggesting tighter control of target properties.","The discrete structure tokens embed into a 2D graph transformer and improve QM9 property prediction for HOMO, LUMO, and their gap, so the tokenizer is useful beyond generation.","The representation is not tied to one line notation; switching from SMILES to SELFIES keeps validity and stability nearly unchanged, so any line notation can carry the 3D tokens.","A direct practical consequence of the speed advantage is that screening large candidate libraries by generating 3D structures becomes feasible at much lower sampling cost than diffusion-based generation."],"supporting_citations":[{"why":"Supplies the VQ-VAE objective and codebook learning that turns continuous descriptors into discrete tokens.","marker":"Van Den Oord et al., 2017"},{"why":"Defines SMILES, the 2D line notation whose token order the spherical coordinates are appended to in the SMILES variant.","marker":"Weininger, 1988"},{"why":"Defines SELFIES, the line notation used for the main generation experiments.","marker":"Krenn et al., 2020"},{"why":"Supplies the GPT-2 architecture and autoregressive next-token objective, plus the KV-cache acceleration used for sampling.","marker":"Radford et al., 2019"},{"why":"Provides the EDM diffusion baseline, the lookup-table validity metric, and the conditional generation protocol the paper follows.","marker":"Hoogeboom et al., 2022"},{"why":"GeoBFN is the previous state-of-the-art conditional generation baseline the paper compares against.","marker":"Song et al., 2023"},{"why":"Introduces the VQ-VAE structure-alphabet idea for proteins that the atom-level tokenizer adapts to small molecules.","marker":"van Kempen et al., 2022"},{"why":"Symphony is the autoregressive baseline whose metric tables and comparison numbers the paper reuses.","marker":"Daigavane et al., 2023"}],"fun_headline_variants":["3D molecules as text tokens, 28x faster generation","Spherical tokenization turns 3D chemistry into language modeling","VQ-VAE tokens let GPT-2 spin up valid 3D molecules","SE(3)-invariant tokens make 3D generation a next-word task","Molecule shapes tokenized for fast, stable GPT-2 generation"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The construction assumes that for every atom, its focal atom and two reference atoms are not collinear; when they are, the local spherical coordinates are undefined and the whole tokenization has nothing to attach to the atom token.","fun_headline_variants_meta":{"raw":{"variants":["3D molecules as text tokens, 28x faster generation","Spherical tokenization turns 3D chemistry into language modeling","VQ-VAE tokens let GPT-2 spin up valid 3D molecules","SE(3)-invariant tokens make 3D generation a next-word task","Molecule shapes tokenized for fast, stable GPT-2 generation"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000205,"raw_usage":{"total_tokens":1464,"prompt_tokens":1086,"completion_tokens":378,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":702,"completion_tokens_details":{"reasoning_tokens":283}},"tokens_in":702,"tokens_out":378,"duration_ms":3851,"temperature":1.0,"reasoning_tokens":283,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T04:16:56.978693+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the tokenizer's descriptor extraction on a linear triatomic molecule such as CO₂: at atoms where the focal atom and its two reference atoms are collinear, the Gram-Schmidt denominator in the construction of $e_2$ becomes exactly zero, so no valid $(d_i, \\theta_i, \\varphi_i)$ descriptor exists and the pipeline either crashes, assigns an arbitrary frame, or fails to reconstruct the molecule.","supporting_citations":[{"cited_title":"SMILES , a chemical language and information system","cited_arxiv_id":null,"evidence_quote":"Defines SMILES, the 2D line notation whose token order the spherical coordinates are appended to in the SMILES variant."},{"cited_title":"Language models are unsupervised multitask learners","cited_arxiv_id":null,"evidence_quote":"Supplies the GPT-2 architecture and autoregressive next-token objective, plus the KV-cache acceleration used for sampling."},{"cited_title":"Equivariant diffusion for molecule generation in 3D","cited_arxiv_id":null,"evidence_quote":"Provides the EDM diffusion baseline, the lookup-table validity metric, and the conditional generation protocol the paper follows."},{"cited_title":"Symphony: S ymmetry-equivariant point-centered spherical harmonics for 3D molecule generation","cited_arxiv_id":null,"evidence_quote":"Symphony is the autoregressive baseline whose metric tables and comparison numbers the paper reuses."}],"review_version":1}