REVIEW 4 major objections 5 minor 60 references
How Well Can Frontier Large Language Models Generate Structures? High Quality Prediction of Molecular Geometries with Help from Fine-Tuning
T0 review · 4 major / 5 minor · reviewed 2026-08-02 · deepseek-v4-flash
Pith's one-line read A large language model fine-tuned on Z-matrix coordinate strings can predict molecular geometries and conformer ensembles more accurately than specialized deep learning models, with the Z-matrix representation carrying most of the advantage
desk verdict The QM9 Z-matrix result is a genuine, useful advance, but the drug-like superiority claim rests on test-set temperature tuning and the language-retention section is unfinished. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The Fenske-Hall Z-matrix: a compact internal-coordinate text format in which each atom's position is defined by a distance, an angle, and a dihedral relative to previously listed atoms. Because it is built from relative rather than absolute coordinates, it is invariant to overall translation and rotation and encodes chemical motifs (bond lengths, angles) as local relational patterns the model can memorize and recombine. The other working part is supervised fine-tuning with Low-Rank Adapters on alpaca-style prompt-response pairs, followed by sampling at temperature/top-p values tuned per task.
What would settle it
Re-run the GEOM-QM9 evaluation scoring every generation, assigning all invalid or wrong-atom-count outputs a maximal error (or treating them as coverage misses), and compare the resulting GeomLlama numbers to DMCG. If the gap closes or reverses, the claimed superiority depends on excluding failures.
Extended reading notes
Core claim
The central claim is that supervised fine-tuning turns an LLM into a competitive molecular conformer generator, and that the Z-matrix representation is the reason it works well. Trained on GEOM-QM9 and GEOM-Drugs with prompt-response pairs (SMILES in, Fenske-Hall Z-matrix out), the fine-tuned GeomLlama model reaches a MAT mean of 0.133 Å versus 0.148 Å for DMCG on GEOM-QM9, and on GEOM-Drugs with temperature-1.4 sampling reaches MAT 0.685 Å versus 0.722 Å for DMCG, with coverage comparable to or better than DMCG. This is achieved with no graph-specific architecture, only token prediction over text, using LoRA fine-tuning and sampling hyperparameters (temperature, top-p) as the conformer-dive
Load-bearing premise
The headline RMSD numbers are computed only for the roughly 90–97% of GeomLlama generations that pass syntax, atom-count, and graph-assignment checks; if the excluded failures are systematically the hardest molecules, real-world accuracy is lower than reported.
Editorial extensions
If this is right
- The Z-matrix representation is a better 'language of geometry' for LLM fine-tuning than Cartesian coordinates: on GEOM-QM9 the Z-matrix model beats DMCG on MAT and nearly matches COV, and on GEOM-Drugs temperature-tuned Z-matrix sampling beats DMCG on both metrics.
- Fine-tuning on geometry alone degrades language performance, but adding roughly 4% alpaca instruction data preserves most natural-language ability while keeping conformer quality, so geometry generation can be a removable LoRA add-on.
- SMILES-to-geometry generation works out of distribution: geometries for SmileyLlama-generated Lipinski-rule molecules are closer to the xTB-optimized minimum than RDKit's, in both energy and RMSD.
- Frontier LLMs without fine-tuning are not reliable geometry generators (very low atom-count validity), so the practical capability comes from fine-tuning, not from pre-trained chemical knowledge.
Reading between the lines
- If invalid generations were counted as worst-case errors rather than excluded, the reported accuracy gap over specialized models might shrink; a fair comparison should report validity-adjusted worst-case RMSD.
- The Z-matrix grammar insight suggests LLMs might also excel at other internal-coordinate or relational text encodings, such as torsional-angle representations of protein backbones, not just Cartesian point clouds.
- Temperature scanning as a conformer-sampling knob is a capability unique to LLMs; one could build an adaptive sampler that raises temperature until coverage saturates, using validity checks as a stopping rule.
- Because the model retains language skills, a single LLM could interleave chemistry and natural-language tasks, enabling agents that discuss and generate structures in the same session.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper investigates whether fine-tuned LLMs can generate molecular geometries from SMILES strings, comparing Cartesian XYZ and Z-matrix representations. The authors fine-tune Llama-3.1 and Qwen models on QM9, GEOM-QM9, and GEOM-Drugs datasets, and report that Z-matrix fine-tuning yields strong performance on single-structure prediction (Table 1) and on multi-conformer generation (Tables 2 and 3), competitive with or better than specialized deep learning baselines such as DMCG. They also demonstrate that mixing a small amount of natural-language instruction data (Alpaca) mitigates catastrophic forgetting, and they evaluate out-of-distribution drug-like molecules generated by their SmileyLlama model.
Significance. If the results hold, the paper makes a useful empirical contribution: it shows that a generic LLM fine-tuned on text-formatted Z-matrices can achieve state-of-the-art conformer-generation accuracy on standard benchmarks, while also preserving general language ability through pseudorehearsal. The release of training data, prompts, and code on GitHub is a concrete strength that supports reproducibility. The comparison against strong baselines (DMCG, GeoMol, ConfGF) on GEOM-QM9 and GEOM-Drugs is valuable, and the finding that Z-matrices are a better token grammar than XYZ coordinates is an interesting and falsifiable claim. However, the headline claim of outperforming specialized models for drug-like molecules rests on test-set-selected inference temperatures in Table 3, and several sections contain unfinished placeholder text, which limits the current reliability of the paper as a whole.
major comments (4)
- [Section 3.3, Table 3] The 'GeomLlama (T-opt)' row (MAT 0.685, COV 94.07) is selected by sweeping temperature on the GEOM-Drugs test set, as stated in Section 3.3 and confirmed by Supplementary Figure S2 labeled 'Temperature dependence of the GEOMLlama predictions on the GEOM-Drugs test data.' At the training temperature T=1.0, GeomLlama (zmat) gives MAT 0.7247 and COV 91.02, which is slightly worse than DMCG on MAT (0.7220) and much worse on COV (96.52). Therefore the paper's central claim that fine-tuned LLMs outperform specialized deep learning models for drug-like molecules is not established by an out-of-sample comparison. The authors should either select T on a validation split and report the corresponding test performance, or downgrade the drug-like claim to 'competitive' rather than 'outperforming.'
- [Table 1 and Section 2.4] The RMSD statistics in Table 1 are computed only for generations passing syntax, atom-count, and RDKit graph-assignment checks; for the best model, GeomLlama-8B (Zmat), Graph % is 89.6, meaning about 10% of generations are excluded. If excluded molecules are systematically harder (e.g., strained rings, stereochemistry), the reported mean/median RMSDs overstate real-world accuracy. The brute-force recovery in Table S1 is a useful mitigation and should be integrated into the main table or clearly stated alongside the headline numbers. Currently the text says 'RMSDs are only reported for successful generation' without quantifying the bias risk.
- [Section 3.5 and Supplementary Table S2] The claim that pseudorehearsal 'retains nearly all of its language modeling ability' is not fully supported yet. The final sentence of Section 3.5 reads: 'To put this into perspective, the Z-matrix trained GeomLlama’s natural language abilities are in the neighborhood of the comparably sized Llama 2-7B model10.to be updated with better results.' This is an explicit placeholder indicating that the language-retention results are still being revised. The reader cannot verify the strength of the pseudorehearsal claim from the current text. This should be completed before acceptance.
- [Section 3.4 and Figure 3] The out-of-distribution analysis with SmileyLlama (Rule-of-Four and Rule-of-Five) is presented as evidence of robustness, but the failure handling is incomplete. For Rule-of-Four, 84/1000 molecules fail to generate a valid Z-matrix and a further ~2% fail GFN2-xTB convergence; the analysis then reports results on the remaining ~90%. The paper should report the performance of RDKit on the same subset and discuss whether the exclusion changes the comparison. Also, the number of molecules used for the RMSD/energy comparison is not stated precisely (only percentages), making it difficult to assess statistical significance.
minor comments (5)
- [Section 2.2] Typo: 'temperatuer' should be 'temperature'. Also, the phrase 'a learning rate somewhere between 2−3×10−4' is imprecise; please give the exact value.
- [Section 4] The phrase 'when using contemporary inference frameworks such as vLLM?' contains a stray question mark. Please remove it or provide a proper citation.
- [Supplementary Figure S2] The figure caption says 'Top-P of 0.95' but it is unclear whether the temperature sweep was performed with top_p held at 0.95 for both XYZ and Z-matrix panels. Please clarify in the caption.
- [References] Reference [15] and [43] appear to be the same paper (Direct Molecular Conformation Generation); please consolidate or clarify why they are cited separately.
- [Section 3.1, Table 1] For the 'Graph %' column, the text says RDKit embedding failures are classified as 'wrong graph'; however, the column is defined as the percentage of correct-atom outputs for which RDKit can capture the graph. Please unify the definitions across the text, table, and Supplementary Table S1.
Circularity Check
Empirical benchmark with no circular derivation; the T-opt GEOM-Drugs row uses a test-set-selected temperature, a statistical-validity caveat rather than a circular step.
full rationale
This paper contains no derivation chain in which a prediction is equivalent to an input by construction. Its claims are empirical measurements (RMSD, COV, MAT) of a fine-tuned LLM (GeomLlama) on held-out, molecule-based splits of QM9, GEOM-QM9, and GEOM-Drugs, compared against external ground-truth conformer ensembles produced by CREST/GFN2-xTB and against published numbers for DMCG and other models (Tables 2 and 3). The metrics align predicted coordinates with reference coordinates from the benchmark, so the target quantity is never defined in terms of the model's own output or a fitted parameter. The Z-matrix-vs-Cartesian conclusion is an empirical comparison of two prompt formats on the same held-out data (Table 1), not an ansatz imported via citation, and no uniqueness theorem is invoked. The only author-self-citation used operationally is SmileyLlama (ref. 6), which generates SMILES strings as test inputs for Section 3.4; it supplies neither fitted parameters nor ground-truth labels for the geometry claims, so it is not load-bearing. The one legitimate concern is statistical, not circular: in Section 3.3 the 'optimal' temperature T=1.4 is selected from the GEOM-Drugs test set (Supplementary Figure S2 is explicitly the 'temperature dependence ... on the GEOM-Drugs test data'), and the resulting 'GeomLlama (T-opt)' row of Table 3 is the value used to claim drug-like outperformance of DMCG; at the training temperature T=1.0 the same model's MAT is 0.7247, slightly worse than DMCG's 0.7220. Choosing a hyperparameter on the evaluation set inflates the reported comparison, but the reported MAT is still a measured outcome of actual generations, not a quantity equal to its own input by construction. Because the central QM9 result (Table 2) is obtained at the training temperature on a held-out split and is self-contained against external benchmarks, the circularity score is appropriately low.
Assumptions & free parameters
free parameters (4)
- inference temperature T =
1.4 (GEOM-Drugs, T-opt row)
- alpaca mixing fraction =
~4% of training examples
- coordinate rounding =
3 decimal places (strict)
- top_p (nucleus sampling) =
0.95
assumptions (6)
- domain assumption QM9, GEOM-QM9, GEOM-Drugs datasets provide correct reference geometries and conformer distributions
- domain assumption OpenBabel's Fenske-Hall Z-matrix conversion preserves the 3D geometry for model training
- domain assumption RDKit RDDetermineBonds correctly identifies molecular graphs for RMSD calculation
- standard math Kabsch/Hungarian RMSD is a valid measure of geometric similarity
- domain assumption GFN2-xTB local minima are a valid reference for the out-of-distribution drug-like test
- domain assumption LoRA supervised fine-tuning is an effective adaptation method for this task
Cite this review
Pith. "Pith review of How Well Can Frontier Large Language Models Generate Structures? High Quality Prediction of Molecular Geometries with Help from Fine-Tuning." pith.science (2026). https://pith.science/paper/AU76OQJP
@misc{pith2026260713350,
author = {Pith},
title = {Pith review of: How Well Can Frontier Large Language Models Generate Structures? High Quality Prediction of Molecular Geometries with Help from Fine-Tuning},
year = {2026},
howpublished = {\url{https://pith.science/paper/AU76OQJP}},
note = {Machine review of arXiv:2607.13350}
}
read the original abstract
The power of Large Language Models (LLMs) has led us to investigate how they might be fine-tuned for learning the "language of molecular geometry". The fine-tuning of the LLMs using Cartesian coordinates or Z-matrices provides an extremely simple method for accurately predicting equilibrium structures and diverse sets of conformers of small organic and drug-like molecules with excellent accuracy and outperforming specialized deep learning models. While the most common representation of molecular geometries are Cartesian coordinates performs adequately, we find that the inherent invariances and relational nature of geometries represented as Z-matrices provides a better grammar for LLM adaptation. Finally, we show that enhancing an LLMs capabilities for robust prediction of small molecule geometries still retains nearly all of its pre-trained language abilities by randomly mixing in small quantities of natural language prompt-response pairs into the fine-tuning.
Figures
Figures from the paper (1 more)
Reference graph
Works this paper leans on
-
[1]
Transactions on Machine Learning Research , issn=
Direct Molecular Conformation Generation , author=. Transactions on Machine Learning Research , issn=. 2022 , url=
2022
-
[2]
Veber, Daniel F. and Johnson, Stephen R. and Cheng, Hung-Yuan and Smith, Brian R. and Ward, Keith W. and Kopple, Kenneth D. , urldate =. Molecular Properties That Influence the Oral Bioavailability of Drug Candidates , volume =. 2002 , journal =. doi:10.1021/jm020017n , abstract =
-
[3]
Christopher A Lipinski and Franco Lombardo and Beryl W Dominy and Paul J Feeney , keywords =. Experimental and computational approaches to estimate solubility and permeability in drug discovery and development settings1PII of original article: S0169-409X(96)00423-1. The article was originally published in Advanced Drug Delivery Reviews 23 (1997) 3–25.1 , ...
-
[4]
2023 , eprint=
Llama 2: Open Foundation and Fine-Tuned Chat Models , author=. 2023 , eprint=
2023
-
[5]
Quantum Chemistry Structures and Properties of 134 Kilo Molecules , author =. Scientific Data , volume =. doi:10.1038/sdata.2014.22 , urldate =
-
[6]
doi:10.1021/ci00057a005 , urldate =
Weininger, David , year = 1988, month = feb, journal =. doi:10.1021/ci00057a005 , urldate =
-
[7]
Axelrod, Simon and. Scientific Data , volume =. doi:10.1038/s41597-022-01288-4 , urldate =
-
[8]
Physical Chemistry Chemical Physics , volume =
Automated Exploration of the Low-Energy Chemical Space with Fast Quantum Chemical Methods , author =. Physical Chemistry Chemical Physics , volume =. doi:10.1039/C9CP06869D , urldate =
Show all 60 references
-
[9]
and Khashabi, Daniel and Hajishirzi, Hannaneh
Wang, Yizhong and Kordi, Yeganeh and Mishra, Swaroop and Liu, Alisa and Smith, Noah A. and Khashabi, Daniel and Hajishirzi, Hannaneh. Self-Instruct: Aligning Language Models with Self-Generated Instructions. Proceedings of the 61st Annual Meeting of the Association for Computa...
2023 doi
-
[10]
Hashimoto , title =
Rohan Taori and Ishaan Gulrajani and Tianyi Zhang and Yann Dubois and Xuechen Li and Carlos Guestrin and Percy Liang and Tatsunori B. Hashimoto , title =. GitHub repository , howpublished =. 2023 , publisher =
2023
-
[11]
AI for Accelerated Materials Design - NeurIPS 2023 Workshop , year=
Fine-Tuned Language Models Generate Stable Inorganic Materials as Text , author=. AI for Accelerated Materials Design - NeurIPS 2023 Workshop , year=
2023
-
[12]
and Banck, Michael and James, Craig A
O'Boyle, Noel M. and Banck, Michael and James, Craig A. and Morley, Chris and Vandermeersch, Tim and Hutchison, Geoffrey R. , year = 2011, month = oct, journal =. Open. doi:10.1186/1758-2946-3-33 , urldate =
2011 doi
-
[13]
RDKit: Open-Source Cheminformatics Software , url =
Landrum, Greg , biburl =. RDKit: Open-Source Cheminformatics Software , url =
-
[14]
Molecular
Mansimov, Elman and Mahmood, Omar and Kang, Seokho and Cho, Kyunghyun , year = 2019, month = dec, journal =. Molecular. doi:10.1038/s41598-019-56773-5 , urldate =
2019 doi
-
[15]
Proceedings of the 37th International Conference on Machine Learning , pages =
A Generative Model for Molecular Distance Geometry , author =. Proceedings of the 37th International Conference on Machine Learning , pages =. 2020 , editor =
2020
-
[16]
Learning
Xu, Minkai and Luo, Shitong and Bengio, Yoshua and Peng, Jian and Tang, Jian , year = 2020, month = oct, urldate =. Learning. International
2020
-
[17]
Learning
Shi, Chence and Luo, Shitong and Xu, Minkai and Tang, Jian , year = 2021, month = jul, pages =. Learning. Proceedings of the 38th
2021
-
[18]
URL https://github.com/axolotl-ai-cloud/axolotl/tree/main , url =
axolotl. URL https://github.com/axolotl-ai-cloud/axolotl/tree/main , url =
-
[19]
Advances in neural information processing systems , volume=
Language models are few-shot learners , author=. Advances in neural information processing systems , volume=
-
[20]
2026 , url =
Microsoft , title =. 2026 , url =
2026
-
[21]
2024 , url =
Anthropic , title =. 2024 , url =
2024
-
[22]
Advances in neural information processing systems , volume=
Geomol: Torsional geometric generation of molecular 3d conformer ensembles , author=. Advances in neural information processing systems , volume=
-
[23]
Transactions on Machine Learning Research , year=
Direct Molecular Conformation Generation , author=. Transactions on Machine Learning Research , year=
-
[24]
, title =
Halgren, Thomas A. , title =. Journal of Computational Chemistry , volume =
-
[25]
2019 , eprint=
Continuous Graph Flow , author=. 2019 , eprint=
2019
-
[26]
2024 , eprint=
A Thorough Examination of Decoding Methods in the Era of LLMs , author=. 2024 , eprint=
2024
-
[27]
2018 , eprint=
Breaking the Beam Search Curse: A Study of (Re-)Scoring Methods and Stopping Criteria for Neural Machine Translation , author=. 2018 , eprint=
2018
-
[28]
and Fenske, Richard F
Hall, Michael B. and Fenske, Richard F. , title =. Inorganic Chemistry , volume =. 1972 , type =. doi:10.1021/ic50110a022 , url =
1972 doi
-
[29]
The Llama 3 Herd of Models , url =
Dubey, Abhimanyu and Jauhri, Abhinav and Pandey, Abhinav and Kadian, Abhishek and Al-Dahle, Ahmad and Letman, Aiesha and Mathur, Akhil and Schelten, Alan and Yang, Amy and Fan, Angela and Goyal, Anirudh and Hartshorn, Anthony and Yang, Aobo and Mitra, Archi and Sravankumar, Ar...
-
[30]
International Conference on Learning Representations , year=
Decoupled Weight Decay Regularization , author=. International Conference on Learning Representations , year=
-
[31]
and Wang, Yingze and Sawyer, Jacob M
Sun, Kunyang and Bagni, Dorian and Cavanagh, Joseph M. and Wang, Yingze and Sawyer, Jacob M. and Zhou, Bo and Gritsevskiy, Andrew and Zhang, Oufan and. ACS Central Science , volume =. doi:10.1021/acscentsci.5c01285 , urldate =
-
[32]
and Sun, Kunyang and Gritsevskiy, Andrew and Bagni, Dorian and Wang, Yingze and Bannister, Thomas D
Cavanagh, Joseph M. and Sun, Kunyang and Gritsevskiy, Andrew and Bagni, Dorian and Wang, Yingze and Bannister, Thomas D. and. doi:10.48550/arXiv.2409.02231 , urldate =. arXiv , keywords =:2409.02231 , primaryclass =
-
[33]
Baker and Ziqi Chen and Xia Ning and Huan Sun , booktitle=
Botao Yu and Frazier N. Baker and Ziqi Chen and Xia Ning and Huan Sun , booktitle=. Lla. 2024 , url=
2024
-
[34]
Translation between
Edwards, Carl and Lai, Tuan and Ros, Kevin and Honke, Garrett and Cho, Kyunghyun and Ji, Heng , year =. Translation between. Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing , publisher =
2022
-
[35]
Dean and Kulik, Heather and
Liu, Yunsheng and Cavanagh, Joseph and Sun, Kunyang and Toney, Jacob and Yuan, Chung-Yueh and Smith, Andrew and St Michel II, Roland and Graggs, Paul and Toste, F. Dean and Kulik, Heather and. Exploring. ChemRxiv , volume =. doi:10.26434/chemrxiv-2025-hm3zb , urldate =
2025 doi
-
[36]
Language Models Can Generate Molecules, Materials, and Protein Binding Sites Directly in Three Dimensions as
Flam-Shepherd, Daniel and Aspuru-Guzik, Alán , date =. Language Models Can Generate Molecules, Materials, and Protein Binding Sites Directly in Three Dimensions as. doi:10.48550/arXiv.2305.05708 , url =. 2305.05708 , eprinttype =
- [37]
-
[38]
From Digits to Decisions: How Tokenization Impacts Arithmetic in LLMs , author=
-
[39]
2024 , eprint=
LoRA Learns Less and Forgets Less , author=. 2024 , eprint=
2024
-
[40]
Connection Science , volume =
Catastrophic Forgetting, Rehearsal and Pseudorehearsal , author =. Connection Science , volume =. https://doi.org/10.1080/09540099550039318 , pages =
-
[41]
Journal of Chemical Theory and Computation , publisher =
Bannwarth, Christoph and Ehlert, Sebastian and Grimme, Stefan , month = mar, year =. Journal of Chemical Theory and Computation , publisher =. doi:10.1021/acs.jctc.8b01176 , number =
-
[42]
and Gomes, Joseph and Geniesse, Caleb and Pappu, Aneesh S
Wu, Zhenqin and Ramsundar, Bharath and Feinberg, Evan N. and Gomes, Joseph and Geniesse, Caleb and Pappu, Aneesh S. and Leswing, Karl and Pande, Vijay , title =. Chemical Science , volume =. 2018 , month =. doi:10.1039/c7sc02664a , url =
2018 doi
-
[43]
and Deems, Stephen and Furlani, Thomas R
Boerner, Timothy J. and Deems, Stephen and Furlani, Thomas R. and Knuth, Shelley L. and Towns, John , title =. Practice and Experience in Advanced Research Computing 2023: Computing for the Common Good , pages =. 2023 , isbn =. doi:10.1145/3569951.3597559 , abstract =
2023
-
[44]
2026 , month = mar, url =
2026
-
[45]
Acta Crystallographica Section A , volume =
A solution for the best rotation to relate two sets of vectors , author =. Acta Crystallographica Section A , volume =. 1976 , publisher =
1976
-
[46]
Digital Discovery , volume =
Nikitin, Filipp and Dunn, Ian and Koes, David Ryan and Isayev, Olexandr , title =. Digital Discovery , volume =. 2025 , month =. doi:10.1039/d5dd00206k , url =
2025 doi
-
[47]
2020 , url =
AICures: Machine Learning Challenge for Drug Discovery and Repurposing , author =. 2020 , url =
2020
-
[48]
Acta Crystallographica Section A , volume =
A discussion of the solution for the best rotation to relate two sets of vectors , author =. Acta Crystallographica Section A , volume =. 1978 , publisher =
1978
-
[49]
Naval Research Logistics Quarterly , volume =
The Hungarian method for the assignment problem , author =. Naval Research Logistics Quarterly , volume =. 1955 , publisher =
1955
-
[50]
2025 , note =
ChatGPT , author =. 2025 , note =
2025
-
[51]
2026 , eprint=
Agents' Last Exam , author=. 2026 , eprint=
2026
-
[52]
doi:10.5281/zenodo.12608602 , url =
Gao, Leo and Tow, Jonathan and Abbasi, Baber and Biderman, Stella and Black, Sid and DiPofi, Anthony and Foster, Charles and Golding, Laurence and Hsu, Jeffrey and Le Noac'h, Alain and Li, Haonan and McDonell, Kyle and Muennighoff, Niklas and Ociepa, Chris and Phang, Jason and...
-
[53]
and Gardner, Matt
Welbl, Johannes and Liu, Nelson F. and Gardner, Matt. Crowdsourcing Multiple Choice Science Questions. Proceedings of the 3rd Workshop on Noisy User-generated Text. 2017. doi:10.18653/v1/W17-4413
2017 doi
-
[54]
CoRR , volume =
Peter Clark and Isaac Cowhey and Oren Etzioni and Tushar Khot and Ashish Sabharwal and Carissa Schoenick and Oyvind Tafjord , title =. CoRR , volume =. 2018 , url =. 1803.05457 , timestamp =
2018 arXiv
-
[55]
B ool Q : Exploring the Surprising Difficulty of Natural Yes/No Questions
Clark, Christopher and Lee, Kenton and Chang, Ming-Wei and Kwiatkowski, Tom and Collins, Michael and Toutanova, Kristina. B ool Q : Exploring the Surprising Difficulty of Natural Yes/No Questions. Proceedings of the 2019 Conference of the North A merican Chapter of the Associa...
2019 doi
-
[56]
The LAMBADA dataset: Word prediction requiring a broad discourse context
Paperno, Denis and Kruszewski, Germ \'a n and Lazaridou, Angeliki and Pham, Ngoc Quan and Bernardi, Raffaella and Pezzelle, Sandro and Baroni, Marco and Boleda, Gemma and Fern \'a ndez, Raquel. The LAMBADA dataset: Word prediction requiring a broad discourse context. Proceedin...
2016 doi
-
[57]
H ella S wag: Can a Machine Really Finish Your Sentence?
Zellers, Rowan and Holtzman, Ari and Bisk, Yonatan and Farhadi, Ali and Choi, Yejin. H ella S wag: Can a Machine Really Finish Your Sentence?. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 2019. doi:10.18653/v1/P19-1472
2019 doi
-
[58]
Thirty-Fourth AAAI Conference on Artificial Intelligence , year =
Yonatan Bisk and Rowan Zellers and Ronan Le Bras and Jianfeng Gao and Yejin Choi , title =. Thirty-Fourth AAAI Conference on Artificial Intelligence , year =
-
[59]
Winogrande: An adversarial winograd schema challenge at scale , volume =
Sakaguchi, Keisuke and Bras, Ronan Le and Bhagavatula, Chandra and Choi, Yejin , journal =. Winogrande: An adversarial winograd schema challenge at scale , volume =
-
[60]
EMNLP , year=
Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answering , author=. EMNLP , year=
Reviewed August 2, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.