{"id":"d1527b41-6950-4e0d-8773-cc07aece20b4","arxiv_id":"2506.19834","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"S23D, a standard LLaMA-style transformer with fixed linear attention biases based on shortest-path graph distances, sets state-of-the-art recall on GEOM-DRUGS at 24.8 million parameters.","lead":"A standard transformer with ALiBi-style linear attention biases over molecular graph distances generates molecular conformations at state-of-the-art recall on GEOM-DRUGS with 25 million parameters, outperforming a 64 million parameter baseline. A generalist might read it because it suggests simple positional encodings can replace expensive equivariant bias in molecular generative models.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Cross-model recall advantage is real per Table 1, but the paper does not isolate the linear-bias PE from the many other S23D/MCF differences; the internal PE ablation cannot carry the attribution in the abstract.","rationale":"We agree with the reader's conditional assessment. The empirical headline in Table 1 is plausible, and the internal ablation is a useful sanity check, but the paper's causal claim that the linear-bias PE is the ingredient that removes the need for a 64M model is underdetermined by the provided comparisons. The load-bearing risk is confounding: many components changed between S23D and MCF, and the within-S23D PE ablation cannot control for those. A second, lesser risk is statistical: a 0.6 pp coverage and 0.015 Å AMR gap reported once without seeds or error bars may be within training noise. These are addressable experimentally, so the paper deserves CONDITIONAL rather than REJECT; we keep the reader's verdict unchanged. We credit the authors for including the PE ablation and detailed architecture and training descriptions, which make the missing controlled cross-model test very concrete.","tokens_in":20292,"tokens_out":5377,"duration_ms":57349,"concrete_test":"In the released MCF codebase, retrain the MCF-B 64M model with the S23D ALiBi PE (Eq. 6, same head slopes and shortest-path distances) replacing the Laplacian-eigenvector PE, keeping the PerceiverIO backbone, noise schedule, loss, data pipeline, 20 conformers per molecule, and 750k-step training budget exactly as in the MCF paper. Run at least three seeds for MCF-B, MCF-B+ALiBi, and S23D-B-1/13 (M) with identical evaluation. If MCF-B+ALiBi reaches or exceeds S23D-B recall (84.6 COV, 0.412 AMR), the PE attribution survives; if it stays at MCF-B levels, the Table 1 gap is driven by other S23D modifications.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that replacing the eigenvector PE with an ALiBi-style shortest-path bias is what lets a 25M non-equivariant transformer beat the 64M MCF-B on GEOM-DRUGS recall. Table 1 supports the raw comparison (S23D-B-1/13 (M): COV-R 84.6, AMR-R 0.412 vs MCF-B: 84.0, 0.427), but the attribution is not supported. S23D and MCF differ simultaneously in backbone (LLaMA vs PerceiverIO), the split graph-encoder/structural-module design, two-stage hydrogen-free pretraining, O(3) augmentation, coordinate encoding, masking, and training budget. Appendix B.1.1 varies the PE only inside S23D-S, and the one S23D-B eigenvector run is single-stage without hydrogens, so it tests whether ALiBi helps S23D, not whether ALiBi explains the cross-model gap against MCF-B. Moreover, the margin is small (0.6 pp COV, 0.015 Å AMR) and no seeded repeats or error bars are reported, so run-to-run noise could erase the advantage. The concrete test should replace MCF's eigenvector PE with S23D's shortest-path linear bias in the original MCF code, holding everything else fixed; only if MCF+ALiBi reproduces the S23D-B numbers would the abstract's statement that a well-chosen positional encoding effectively addresses size limitations be established.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper introduces S23D, a non-equivariant diffusion model for molecular conformer generation built from a LLaMA-style transformer backbone. The main technical proposal is a relative positional encoding implemented as a negative attention bias that grows linearly with the shortest-path distance between graph nodes, using head-specific slopes analogous to ALiBi. The model is trained with a two-stage protocol that first pretrains on hydrogen-free molecules and then finetunes on complete molecules, and a post-hoc chirality correction is applied in some variants. On GEOM-DRUGS, the S23D-B model (24.8M parameters) is reported to surpass the 64M-parameter MCF-B baseline on recall coverage and AMR (COV-R 84.6 vs 84.0; AMR-R 0.412 vs 0.427), and with chirality correction it reaches COV-R 87.0, exceeding the 242M MCF-L model on recall. A positional-encoding ablation in Appendix B.1.1 shows that both the ALiBi-style bias and a learnable shortest-path bias outperform Laplacian-eigenvector positional encodings within the S23D architecture. The paper also analyzes the effect of chirality correction on precision metrics and proposes a two-stage training protocol that reduces computational cost.","tokens_in":20667,"tokens_out":15278,"duration_ms":142295,"significance":"If the reported results are reproducible, the paper provides a useful data point that a simple relative positional encoding can make non-equivariant transformers parameter-efficient for conformer generation, and the two-stage training protocol and chirality-correction analysis are practical contributions to the MCG literature. The paper does not ship code, trained models, or seed-averaged results, and the causal attribution of the benchmark gains to the specific positional encoding is not fully isolated from other architectural and training differences. Nevertheless, the question the paper addresses is timely, the comparisons with published MCF and ET-Flow numbers are transparently tabulated, and the internal PE ablation, while not controlling the cross-model comparison, is a step in the right direction. The work is likely to be of interest to the conformer-generation and graph-transformer communities.","major_comments":[{"comment":"The abstract states that a well-chosen positional encoding effectively addresses model-size limitations, but the reported experiments do not isolate the positional encoding from the other differences between S23D and MCF. S23D-B differs from MCF-B in backbone (LLaMA vs PerceiverIO), graph-encoder/structural split, coordinate encoding, O(3) augmentation, masking, and training schedule, so the 0.6 percentage-point COV-R margin (84.6 vs 84.0) and 0.015 Å AMR-R margin in Table 1 cannot be attributed specifically to the linear-bias PE. The ablation in Appendix B.1.1 varies the PE only within S23D-S, and the S23D-B eigenvector run is single-stage and without hydrogens, so it does not hold the rest of the pipeline fixed against MCF. A controlled experiment that replaces MCF's eigenvector PE with this shortest-path bias in the original MCF code, or a clear reframing of the abstract claim to refer to the S23D architecture as a whole, is needed to support the attribution.","section":"Abstract; §4.2, Table 1"},{"comment":"No error bars, confidence intervals, or repeated-seed results are reported for any S23D result. The headline margins over MCF-B are small (0.6 pp COV-R and 0.015 Å AMR-R), and both diffusion training and the RMSD-based evaluation metrics are stochastic. Without variance estimates, the claim that S23D-B 'surpasses' MCF-B cannot be distinguished from run-to-run noise. Please report mean and standard deviation over at least three seeds, or otherwise quantify the noise floor, for the central comparisons in Table 1.","section":"Table 1; Appendix B.1.1"},{"comment":"The ablation shows that a standard learnable Graphormer-style bias performs essentially identically to the ALiBi-style linear bias (COV-R 81.5 vs 81.4; AMR-R 0.468 vs 0.472) at small scale. This undercuts the emphasis in the abstract and contributions on the specific linear-slope schedule as the source of the improvement; the distinguishing result is the improvement over the Laplacian-eigenvector PE, not the particular functional form of the bias. Please either demonstrate an advantage of the fixed linear schedule over the learnable baseline in some setting or revise the contribution claim to focus on the relative-PE-vs-eigenvector-PE comparison.","section":"Appendix B.1.1, Table 3"}],"minor_comments":[{"comment":"The description of the ALiBi slope schedule is ambiguous; please state explicitly how m_head is computed as a function of the number of heads and head index, and confirm that it matches the original ALiBi paper.","section":"Section 3.1.1, Eq. (6)"},{"comment":"The eigenvector positional encoding is described as ordered by ascending eigenvalues in Section 2 but by largest eigenvalues in B.1.1; please reconcile these descriptions.","section":"Section 2 vs B.1.1"},{"comment":"The PE ablation rows are single-stage training with hydrogens, so they are not directly comparable to the two-stage rows in Table 1; please state this explicitly in the table caption.","section":"B.1.1, Table 3 caption"},{"comment":"The QM9 evaluation uses 995 test molecules, with an assumption that MCF's published numbers are on the same subset; please verify this or note the discrepancy.","section":"B.2.1"},{"comment":"The text and Figure 3 refer to S23D-B-1/9 and S23D-S-1/13 configurations that do not appear in Table 2; please correct the model naming, which is likely S23D-B-1/13 and S23D-S-1/9.","section":"B.3.2 and Figure 3"},{"comment":"There is a typo in the noise schedule description: 'β_in (0.0,18.0)' should presumably be 'β_s' or 'β_start'.","section":"Section A.2"},{"comment":"There is a typo in 'atomi coordinate encodings'; it should be 'atom i coordinate encodings'.","section":"Section 3.1.2"},{"comment":"The sentence 'we estimate that further scaling of the S23D-B model will surpass precision metric performance' is speculative and not a result; consider removing it or clearly labeling it as extrapolation.","section":"Section 4.2"},{"comment":"No code or model release is mentioned; please add a data and code availability statement, since the reproducibility of the reported benchmark numbers would be greatly enhanced by releasing the evaluation code and training configurations.","section":"General"}],"recommendation":"major_revision","confidential_remarks":"The raw benchmark numbers in Table 1 appear credible and the PE ablation is a useful internal comparison, but the central attribution claim and the lack of variance reporting make the headline conclusion fragile as currently written. The authors should be encouraged to either run the controlled MCF+ALiBi experiment or soften the abstract to describe what is actually shown. The naming inconsistencies in the appendix and the missing code availability statement should also be fixed. With these revisions, this could become a solid empirical contribution suitable for the journal."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe headline is that S23D reports a genuine empirical result: a 25M-parameter standard transformer with ALiBi-style graph biases beats the 64M MCF-B on GEOM-DRUGS recall (COV 84.6 vs 84.0, AMR 0.412 vs 0.427). I believe the numbers. They're consistent across multiple coordinate encodings, and the chirality-corrected variants push recall to 87.0, which is impressive. The internal ablation in B.1.1 shows the ALiBi PE outperforms the eigenvector PE used by MCF when swapped into S23D, which is a useful sanity check.\n\nWhat's actually new: this is the first demonstration I've seen of ALiBi-style shortest-path biases applied to conformer generation, and it does seem to give a parameter-efficiency advantage. The two-stage hydrogen-free pretraining is a practical speedup, and the chirality correction for non-equivariant models is a fair point about benchmark comparisons. The inference speed measurements are useful for practitioners.\n\nThe soft spot is attribution. S23D differs from MCF in backbone (LLaMA vs PerceiverIO), graph encoder, two-stage training, O(3) augmentation, and training budget. The ablation varies only the PE inside S23D-S, so it can't tell us whether the linear bias alone explains the cross-model gap. The margin is small — 0.6 pp COV, 0.015 Å AMR — and no error bars or seeded repeats are reported. The abstract's claim that 'a well-chosen positional encoding effectively addresses these size limitations' is stronger than the evidence. The clean experiment would be to take MCF's own code, replace its eigenvector PE with S23D's shortest-path linear bias, and see whether the recall gap closes. The authors don't release code, which makes it harder to check.\n\nNone of this is fatal. The raw result is valuable and the paper is honest about limitations (they note the random splits, the chirality limitation, and the saturated QM9 benchmark). I'd send it to review. A serious referee will push for controlled ablations and error bars, but the empirical claim deserves that attention.","headline":"S23D's recall numbers over MCF-B look real as reported, but the paper over-attributes the gain to its positional encoding when other architectural changes are bundled in; still worth peer review.","tokens_in":21175,"tokens_out":2651,"would_cite":true,"duration_ms":24680,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A standard transformer with ALiBi-style linear graph-distance biases reaches higher conformer recall on GEOM-DRUGS at 25M parameters than a 64M non-equivariant baseline.","keywords":["molecular conformer generation","transformer","ALiBi positional encoding","relative positional encoding","diffusion model","GEOM-DRUGS","non-equivariant model","chirality correction"],"falsifier":"Take the S23D-B-1/13 model and training protocol exactly as described but replace the ALiBi shortest-path bias with MCF's eigenvector positional encoding; if recall coverage stays around 84.6% instead of dropping toward the eigenvector-PE results in the ablation, then the linear bias is not the causal ingredient. The complementary check is to add the same linear bias to the MCF architecture without any of the other changes and see whether it reaches S23D-B's numbers.","tokens_in":20101,"feed_emoji":"⚛️","tokens_out":6732,"duration_ms":61801,"temperature":0.7,"pith_summary":"This paper argues that the main reason non-equivariant transformer models for molecular conformer generation need so many parameters is a poorly chosen positional encoding. It introduces S23D, a standard LLaMA-style transformer whose attention subtracts a head-specific slope times the shortest-path distance between atoms, analogous to ALiBi in NLP. At 24.8M parameters it reports recall coverage of 84.6% and AMR of 0.412 on GEOM-DRUGS, beating the 64M MCF baseline at 84.0% and 0.427 without chirality features, and with a post hoc chirality correction it reaches 87.0% recall coverage. If the comparison is right, a cheap fixed graph bias supplies enough inductive bias to make non-equivariant transformers competitive at a fraction of the size.","feed_headline":"A 25M transformer out-recalls a 64M model for molecular conformers","feed_subtitle":"ALiBi-style distance biases in a standard transformer reach top recall on GEOM-DRUGS at 39% of the baseline's size.","key_machinery":"The load-bearing object is the ALiBi-style linear graph bias inserted into every multi-head attention layer. In place of Graphormer's learnable bias lookup, S23D uses $A^{\\mathrm{head}}_{ij} = (h_i W^{\\mathrm{head}}_Q)(h_j W^{\\mathrm{head}}_K)^T/\\sqrt{d_{\\mathrm{head}}} - m_{\\mathrm{head}}\\phi(v_i,v_j)$, where $\\phi(v_i,v_j)$ is the shortest-path distance between atoms in the molecular graph and $m_{\\mathrm{head}}$ follows ALiBi's geometric slope schedule. This fixed bias costs nothing to learn, involves no lookup table, and lets different heads specialize: heads with large slopes can encode graph structure while heads with near-zero slopes can process atom coordinates. The rest of the architecture is a standard LLaMA transformer block split into a graph encoder and a structural subnetwork, with coordinate encodings injected after the first normalization layer and a VPSDE diffusion score network on top.","core_discovery":"The central claim is that a well-chosen relative positional encoding, rather than model scale, is what lets a non-equivariant diffusion transformer compete with larger and more specialized models for conformer generation. The paper implements relative encoding as a negative attention bias linear in graph shortest-path distance, with head-specific slopes taken from the ALiBi geometric sequence. With this bias, S23D-B-1/13 at 25M parameters surpasses MCF-B at 64M parameters on GEOM-DRUGS recall metrics and comes within 0.1% recall coverage of MCF-L at 242M parameters, establishing a new recall SOTA among models under 64M parameters. The paper also shows that a two-stage protocol of hydrogen-free pretraining followed by finetuning on complete molecules is viable, and that a simple RDKit chirality correction improves both recall and precision, narrowing the precision gap relative to the equivariant ET-Flow model.","pith_inferences":["The headline comparison changes several ingredients at once—backbone, graph encoder, two-stage training, O(3) augmentation, coordinate encoders, and training budget—so the paper's attribution of the gain to the positional encoding would only be confirmed by an ablation that changes only the PE while holding the rest fixed.","If the linear shortest-path bias is the active ingredient, it should transfer to equivariant backbones and to other molecular graph tasks; a cheap test would be adding the same bias to a flow-matching or message-passing model and measuring the parameter savings.","The ALiBi slope schedule is copied from language modeling; a graph-specific schedule, for example one that also depends on degree or on the diameter of the molecule, is a natural extension that could improve the already small models.","The authors note that current GEOM benchmark splits are random rather than scaffold-based, so a scaffold-split evaluation would clarify whether the parameter efficiency holds on molecules that are less similar to training data."],"forward_implications":["S23D-B-1/13 (24.8M parameters) reports recall coverage of 84.6% and AMR of 0.412 on GEOM-DRUGS without chirality features, beating the 64M MCF-B baseline at 84.0% COV and 0.427 AMR.","With a post hoc RDKit chirality correction, the same model reaches 87.0% recall coverage and 0.380 AMR, outperforming the 242M MCF-L on recall while using about a tenth of the parameters.","The two-stage hydrogen-free pretraining protocol reduces the number of atom tokens by roughly 44% in the first stage, and a hydrogen-free model already surpasses the previous small-model recall SOTA.","The ALiBi-style linear bias is about 35% faster to train than the learnable shortest-path bias of Graphormer and gives comparable or better metrics in the ablation, and S23D is several times faster per diffusion step than MCF at similar batch sizes.","Chirality correction increases recall coverage by more than 2 points and precision coverage by roughly 6 points for the small model, showing that chirality handling needs to be reported for fair cross-model comparison."],"supporting_citations":[{"why":"GEOM dataset: supplies the reference conformer ensembles and benchmark data on which all metrics are computed.","marker":"[1]"},{"why":"GeoMol: established the train/validation/test splits and conformer-per-molecule counts reused by S23D and the baselines.","marker":"[10]"},{"why":"ET-Flow: the equivariant precision-SOTA baseline whose chirality features and correction step frame the chirality comparison.","marker":"[14]"},{"why":"Torsional Diffusion: established the 0.75 Å coverage threshold and benchmark protocol for GEOM-DRUGS.","marker":"[22]"},{"why":"ALiBi: source of the geometric head-slope schedule used to build the linear graph-distance attention bias.","marker":"[31]"},{"why":"LLaMA: supplies the transformer block that is the backbone of the S23D score network.","marker":"[41]"},{"why":"MCF: the non-equivariant SOTA baseline, provider of the graph-Laplacian eigenvector PE and processed data compared against.","marker":"[43]"},{"why":"Graphormer: provides the learnable shortest-path attention bias that the paper compares with its linear bias in the ablation.","marker":"[47]"}],"fun_headline_variants":["ALiBi-style bias lets 25M transformer beat 64M model","Small transformer, better recall: ALiBi for conformers","Distance-bias attention trumps model scale at 25M","Head-specific slopes cut conformer model size to 25M","Tiny transformer out-recalls 64M via linear biases"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The claim that a well-chosen positional encoding removes the need for a large non-equivariant model rests on the assumption that the other simultaneous differences between S23D and MCF—backbone, graph encoder, two-stage training, augmentation, coordinate encodings, and training budget—are not the actual source of the benchmark improvement.","fun_headline_variants_meta":{"raw":{"variants":["ALiBi-style bias lets 25M transformer beat 64M model","Small transformer, better recall: ALiBi for conformers","Distance-bias attention trumps model scale at 25M","Head-specific slopes cut conformer model size to 25M","Tiny transformer out-recalls 64M via linear biases"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00053,"raw_usage":{"total_tokens":2556,"prompt_tokens":952,"completion_tokens":1604,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":568,"completion_tokens_details":{"reasoning_tokens":1515}},"tokens_in":568,"tokens_out":1604,"duration_ms":12198,"temperature":1.0,"reasoning_tokens":1515,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T18:24:47.416784+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take the S23D-B-1/13 model and training protocol exactly as described but replace the ALiBi shortest-path bias with MCF's eigenvector positional encoding; if recall coverage stays around 84.6% instead of dropping toward the eigenvector-PE results in the ablation, then the linear bias is not the causal ingredient. The complementary check is to add the same linear bias to the MCF architecture without any of the other changes and see whether it reaches S23D-B's numbers.","supporting_citations":[{"cited_title":"GEOM, energy-annotated molecular conformations for property prediction and molecular generation.Scientific Data, 9(185), 2022","cited_arxiv_id":null,"evidence_quote":"GEOM dataset: supplies the reference conformer ensembles and benchmark data on which all metrics are computed."},{"cited_title":"Coley, Regina Barzilay, Klavs F","cited_arxiv_id":null,"evidence_quote":"GeoMol: established the train/validation/test splits and conformer-per-molecule counts reused by S23D and the baselines."},{"cited_title":"ET-Flow: Equivariant flow-matching for molecular conformer generation","cited_arxiv_id":null,"evidence_quote":"ET-Flow: the equivariant precision-SOTA baseline whose chirality features and correction step frame the chirality comparison."},{"cited_title":"Train short, test long: Attention with linear biases enables input length extrapolation","cited_arxiv_id":null,"evidence_quote":"ALiBi: source of the geometric head-slope schedule used to build the linear graph-distance attention bias."},{"cited_title":"Swallowing the bitter pill: Simplified scalable conformer generation","cited_arxiv_id":null,"evidence_quote":"MCF: the non-equivariant SOTA baseline, provider of the graph-Laplacian eigenvector PE and processed data compared against."}],"review_version":2}