A Reasoning-Based Approach to Cryptic Crossword Clue Solving

Martin Andrews; Sam Witteveen

REVIEW 3 major objections 5 minor 36 references

A Reasoning-Based Approach to Cryptic Crossword Clue Solving

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A three-stage LLM pipeline that writes wordplay as Python assertions and verifies them reaches 32.5% exact match on the Cryptonite test set, with a 92% Bayesian probability of beating GPT-4o.

desk verdict A solid, openly-licensed cryptic-solving pipeline that deserves peer review, but the new-SOTA claim rests on 200 samples and is statistically fragile. read the letter →

arxiv 2506.04824 v1 pith:EZSXNFRF submitted 2025-06-05 cs.CL cs.AI

Martin Andrews , Sam Witteveen This is my paper

classification cs.CLcs.AI

keywords crypticcrosswordcluesolvingLLMreasoningformalverificationPythonDSLwordplayCryptonitebenchmarkanswercandidategeneration

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that cryptic crossword clues—sentences that hide a definition plus rule-governed wordplay—can be solved as a formal reasoning task rather than a next-word guessing task. Its system generates twenty answer candidates with a fine-tuned 9-billion-parameter model, proposes ten wordplay explanations per candidate, then asks a second LLM to translate the wordplay into Python assertions that a purpose-built verifier executes line by line. A clue is 'proved' only when every assertion passes, and failed proofs are fed back as hints for up to two rewrites. On the Cryptonite benchmark the full pipeline reaches 32.5% Top-1 exact match on the test set, a new state of the art against updated baselines, with the proved solutions available as human-readable Python. The authors contend that the formalisation-plus-verification stage, not the answer candidate ranker, is what carries the result.

What carries the argument

The load-bearing mechanism is a generate–formalise–verify loop whose formal target is a tiny embedded Python DSL: wordplay is expressed as assert statements over functions such as is_synonym(), is_anagram(), is_homophone(), action_type(), and is_abbreviation(), with Action an enum of indicators (ANAGRAM, REVERSE, INITIALS, and so on). The verifier deconstructs the code's abstract syntax tree and evaluates the assertions line by line, so it can report exactly which wordplay step failed and offer hints ('goes crazy does not suggest ANAGRAM, but crazy does'); those hints feed back to the formalising LLM for up to two rewrite rounds. String manipulations such as reversal and letter stripping are handled directly by Python, while the semantic primitives are LLM calls or nearest-neighbour look-ups against indicator and abbreviation lists. The asymmetry the whole system depends on is that genuine answers give wordplay that compiles into passing assertions, whereas invented wordplay for wrong answers tends to fail at some step.

What would settle it

Take a set of clues with known wrong answers that match the letter pattern, ask the pipeline to generate wordplay for them, and count how often the verifier returns SUCCESS: if a substantial fraction of wrong answers are 'proved', the core asymmetry fails. A more direct check is to compare exact-match accuracy on clues whose answers were proved versus clues that fell back to the frequency pick—if proved answers are not clearly more accurate, the proof step adds no information.

Watch

Extended reading notes

Core claim

The central discovery is that a correct answer to a cryptic clue tends to admit formalisable, verifiable wordplay, while an incorrect candidate does not, and that this asymmetry can be exploited mechanically. Concretely: fine-tune Gemma2-9B to propose answers and wordplay; formalise the wordplay as a Python proof whose assert statements call domain functions (is_synonym, is_anagram, is_homophone, action_type, is_abbreviation); parse the generated proof with the Python AST; execute every assert, returning targeted hint messages on failure; and declare the answer proved when every assertion holds. This loop achieves 32.5% Top-1 exact match on the Cryptonite test set (with 200 samples), versus 27.6% for GPT-4o under matching 5-shot conditions, and a Bayesian IRT estimate of 92% probability that the full system is genuinely better than GPT-4o. The same pipeline with a non-fine-tuned open Gemma2-9B-it formaliser reaches 29.0% on the test set, showing the approach does not depend on the proprietary formaliser.

Load-bearing premise

The whole method rests on the claim that a right answer always yields wordplay that can be written as verifiable Python assertions, while a wrong answer does not—and because the verifier's synonym, homophone, and indicator checks are themselves heuristic LLM or nearest-neighbour guesses, a wrong answer whose wordplay slips past those checks would be falsely 'proved'.

Editorial extensions

If this is right

If the central claim holds, a 9-billion-parameter open-licensed stack can match or beat a much larger proprietary model on Cryptonite by spending more inference-time compute rather than more parameters.
The ablation results imply that neither answer-candidate frequency nor wordplay log-probability can substitute for the formalisation-and-verification stage, so the proof loop is the source of the gain.
Proved answers come with inspectable Python reasoning, so a solver using this system can check the wordplay itself rather than trusting a black-box answer.
Because the formaliser is prompted in context and the verifier is deterministic, the system should improve directly with better candidate coverage and with more test-time compute, as the paper's cumulative candidate analysis suggests (roughly 45% gold in the top 20 bounds overall accuracy).
The partial-correctness experiments show the same pipeline can exploit known grid letters at the 25% level, and that a FastText nearest-neighbour search over a crossword dictionary is a strong option at 50–70% known letters.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A natural extension is to treat the 'proved' label as a precision filter: measure exact-match accuracy on proved versus unproved clues separately, and only trust the answer when the proof succeeds; the paper does not report this split directly.
The weakest components are the semantic primitives, not the Python mechanics, so replacing is_synonym and is_homophone with stronger lexical or learned models should lift accuracy more than improving answer generation once coverage exceeds roughly 45%.
The same generate–formalise–verify recipe could transfer to other NLP puzzles with checkable compositional structure—such as word ladders, riddles, or annotation tasks—where a correct answer yields an inspectable derivation and an incorrect one does not.
The paper's own limitation list implies that an RL agent trained on verifier success would learn to exploit comment-only proofs, conditional assertions, or assert X == False rewrites unless the verifier rejects those shortcuts, so any RL extension must harden verification first.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Short version: this is a real systems contribution, not a breakthrough. It builds an open, mostly local pipeline for cryptic crossword clues: fine-tuned Gemma2-9B models for answer candidates and wordplay, then formalization into Python and an AST verifier with hinting. The code is public, the cost is low, and the ablations show the verification step actually matters over simple logprob ranking. The authors also list known verifier loopholes in Section 4.6 and do not overclaim interpretability. That transparency is worth credit.

The soft spot is the headline empirical claim. The 32.5% vs GPT-4o's 27.6% on the test set comes from 200 sampled clues. With the paper's own reported s.d. of roughly 3.3% per system, the 4.9-point gap is about 1.1 standard errors of the difference, which is not a significant result by conventional standards. On the validation split the ordering reverses: 28.0% vs 29.8%. The Bayesian IRT "92% probability of being better" is a moderate posterior, not a robust finding, and it is computed on that same 200-sample test set. The stress-test note is right: the combined evidence is close to noise. The GPT-4o contamination caveat is also real, and the paper acknowledges it.

Is this fatal? No. The contribution is the pipeline and the open verification approach, not the precise SOTA number. The verifier's semantic primitives are LLM/heuristic functions, so "proof" is not a formal guarantee, again acknowledged. That does not directly inflate accuracy, since exact-match evaluation is against gold answers; it does mean the accuracy gain could partly come from a permissive verifier, and a few hundred samples cannot disentangle that. The Quick-clues result also looks odd (the formalizer underperforms the simple frequency baseline) and deserves a closer look.

I would send this to review. The right request would be: run both splits at 1000 samples, report a paired test or McNemar, and tighten the SOTA language unless it survives. The code and open-license positioning make it worth referee time. I would cite it for the system and the verifier design, not for the headline number.

Referee Report

3 major / 5 minor

Summary. This paper describes a three-stage pipeline for cryptic crossword clue solving: a fine-tuned Gemma2-9B model proposes answer candidates, a second fine-tuned model generates informal wordplay, and a formaliser (Gemini-Flash-1.5 or Gemma2-9B-it) converts the wordplay into Python assertion-based proofs that are checked by a custom AST-based verifier with hinting and retries. On the Cryptonite test split the Gemini-Flash Formaliser achieves 32.5% exact-match accuracy versus 27.6% for GPT-4o, but the validation split shows the reverse ordering (28.0% vs 29.8%). The paper claims a new state-of-the-art result and emphasises that each proved solution is expressed as inspectable Python.

Significance. If the statistical comparison is confirmed on a larger sample, this is a valuable contribution: it shows that open-licensed 9B models, combined with formalisation and verification, can rival much larger proprietary models on a hard language-reasoning benchmark, at low cost, with code and models released. The paper also responsibly reports standard deviations, uses Bayesian IRT, includes ablations, and explicitly documents verifier limitations. The main value is the verification-architecture idea rather than the specific accuracy number.

major comments (3)

[4.3, Table 1] The headline claim that the Gemini-Flash Formaliser establishes a new state-of-the-art result is supported only by a 200-sample test measurement. With the reported standard deviation of approximately 3.3% at n=200, the 4.9-point gap over GPT-4o is about 1.1 standard errors of the difference, and the validation split shows the opposite ordering (28.0% vs 29.8%). The Bayesian IRT statement that the model has a 92% probability of being better is not a substitute for a paired significance test or a larger sample, especially when the two splits disagree. I recommend either reporting a paired McNemar test and/or increasing the sample, or softening the state-of-the-art claim to 'competitive'.
[3.5, 4.6] The paper's central interpretability contribution is the claim that solutions are 'proved' by a verifier, but Section 4.6 explicitly lists four ways the verifier can be satisfied without a valid wordplay proof: comment-only functions, conditional routing around asserts, rewriting failed asserts as assert X == False, and logically disconnected proofs. This is not merely a minor caveat: if invalid proofs are accepted for correct answer candidates, the accuracy gain may come from a permissive heuristic acceptance filter rather than from sound wordplay verification, and the displayed 'proven' reasoning can be misleading. I ask for a quantitative audit: on a random sample of successful proofs, what fraction are genuinely valid under the stated cryptic-crossword conventions, and what fraction exploit these loopholes? The authors can also tighten the AST checks to reject these cases.
[4.2] The system's key assumption, stated in Section 4.2, is that correct answer candidates yield formulisable and verifiable wordplay while incorrect candidates do not. Only a single qualitative example is given. Since the verifier's semantic primitives are themselves LLM or nearest-neighbour heuristics, this assumption is not automatically satisfied. I suggest reporting the verifier acceptance rate split by candidate correctness on a held-out sample. If wrong candidates are also accepted at a non-negligible rate, the mechanism behind the accuracy improvement needs to be re-described.

minor comments (5)

[5] In the Conclusions, 'transferable to a the open-licensedGemma2-it model' should read 'transferable to the open-licensed Gemma2-it model'.
[A.3] The appendix uses 'Rosner' once where 'Rozner' is used elsewhere; please standardise the spelling.
[A.5.4] The example formalisation code contains 'is_synomym' instead of 'is_synonym' in two places.
[4.3] The text refers to 'Gemma-9B FT' in one place while Table 1 and elsewhere use 'Gemma2-9B FT'; please standardise the model name.
[Figure 7] The candidate-generation curves would be easier to interpret with error bars or confidence bands, given the relatively small sample sizes involved.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the headline exact-match accuracy is scored against external gold answers, and no fitted parameter is recycled into the reported result.

full rationale

The paper's central claim is Top-1 exact-match accuracy on the held-out Cryptonite test set, where the answer is compared with the externally provided gold answer. None of the system components—candidate generation, wordplay suggestion, Python formalisation, or the verifier—introduces a fitted parameter that is then reported as a prediction on the test set. The fine-tuned Gemma2 models are trained on Cryptonite training splits and the Wordplay dataset, with test answers scrubbed, so the evaluation is not circular by construction. The verifier's semantic primitives (is_synonym, is_homophone, action_type) are heuristic and partly LLM-based, which weakens the word 'proved' as a formal guarantee, but this is a soundness limitation explicitly acknowledged in Section 4.6, not a circular derivation: a wrong answer that passes the verifier is still counted as incorrect in exact-match evaluation. The self-citations to Andrews & Witteveen (2024, 2025) and the Wordplay dataset are contextual prior-work and training-resource references, not load-bearing uniqueness claims or imported constraints that force the reported result. Statistical fragility of the SOTA claim due to the small 200-sample test size is a correctness and robustness concern, not a circularity concern. Under the strict criteria requiring a quoted equation-level or definition-level reduction, no circular step is present.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

No new physical or conceptual entities are postulated. The Python DSL is a notation, not an invented entity. The main free parameters are pipeline hyperparameters and the implicit synonym-distance threshold.

free parameters (5)

candidate_count_k = 20
Number of answer candidates generated per clue; chosen as a cost/coverage trade-off based on Figure 7, not derived.
wordplay_suggestions_count = 10
Wordplay hypotheses per answer candidate; set by hand in the pipeline.
max_rewrites = 2
Maximum formaliser retries after verifier feedback; chosen by hand.
candidate_generation_temperature = 1.0
Used for answer candidate sampling to increase spread; t=0.5 gave better Top-1 but t=1.0 helped the pipeline overall.
synonym_distance_threshold = not specified
The is_synonym fallback uses an LLM or thesaurus look-up with an implicit threshold; the paper notes setting this 'synonym distance hurdle' is an ongoing challenge (Section 4.5).

assumptions (4)

domain assumption Cryptonite Top-1 exact match on sampled test subsets is a valid measure of cryptic clue solving performance.
Used to define SOTA; sample sizes 1000/200 with reported std devs.
domain assumption The Wordplay dataset's human annotations are correct enough to train wordplay suggestion models.
Fine-tuning of the wordplay model relies on these annotations (Section 3.2).
domain assumption The verifier's semantic predicates (is_synonym via thesaurus/dictionary/LLM, action_type via nearest-neighbour indicator lists) are adequate proxies for cryptic wordplay validity.
The 'proof' status of answers depends on these functions (Section 3.3).
ad hoc to paper A correct answer candidate is more likely to generate formulisable wordplay than an incorrect one.
This is the key assumption stated in Section 4.2; it underlies the filtering power of the verifier.

how reviews work

0 comments

Cite this review

Pith. "Pith review of A Reasoning-Based Approach to Cryptic Crossword Clue Solving." pith.science (2026). https://pith.science/paper/EZSXNFRF

@misc{pith2026250604824,
  author       = {Pith},
  title        = {Pith review of: A Reasoning-Based Approach to Cryptic Crossword Clue Solving},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/EZSXNFRF}},
  note         = {Machine review of arXiv:2506.04824}
}

read the original abstract

Cryptic crossword clues are challenging language tasks for which new test sets are released daily by major newspapers on a global basis. Each cryptic clue contains both the definition of the answer to be placed in the crossword grid (in common with regular crosswords), and 'wordplay' that proves that the answer is correct (i.e. a human solver can be confident that an answer is correct without needing crossing words as confirmation). This work describes an LLM-based reasoning system built from open-licensed components that solves cryptic clues by (i) hypothesising answers; (ii) proposing wordplay explanations; and (iii) using a verifier system that operates on codified reasoning steps. Overall, this system establishes a new state-of-the-art performance on the challenging Cryptonite dataset of clues from The Times and The Telegraph newspapers in the UK. Because each proved solution is expressed in Python, interpretable wordplay reasoning for proven answers is available for inspection.

Figures

Figures reproduced from arXiv: 2506.04824 by the authors.

**Figure 1.** Proving process: answer candidate → wordplay → LLM formalisation • There are decades of solved puzzles (each one containing over 20 clues) from multiple major newspapers available, and new puzzles are published daily. This contrasts with (for instance) IMO/AIME problems, where there is a much lower number of novel problems available. • The method in this work explicitly reveals the reasoning (i.e. validated wordplay… view at source ↗

**Figure 2.** Clue solving illustrations. Answers are in green, definitions in blue (dashed frame), wordplays in orange, and indicators in purple. Further textual examples can be found in Appendix A.1 thwart a strictly formal approach. A more flexible rule-based solver with a manually-crafted probabilistic grammar was introduced in Deits (2015; 2022). Building on the assumption that a clue can usually be split into wordplay and d… view at source ↗

**Figure 3.** An example from the Wordplay dataset (in this wordplay, ()* is an anagram indicator). This clue’s solution is diagrammed in Figure 2b of letters are anagrams of each other. In contrast with the tool-integrated reasoning framework Gou et al. (2024), where an LLM for mathematical problemsolving was fine-tuned on 16,000 examples of formalisation, we found that our light-DSL was able to be used by LLMs based on its in-… view at source ↗

Figures from the paper (2 more)

**Figure 5.** Figure 5: External functions available via In-Context Learning a limiting factor, we subsequently tested an unmodified Gemma2-9B-it model on the same task. In terms of the DSL itself, the back-end to the is synonym and is homophone functions consists of calls to simple language…

**Figure 6.** Figure 6: Illustrative AssertionError responses (with hinting) from the verifier 3.5. Proof Verification with Hinting The system’s verifier must decide whether a given formalisation is valid, and report any errors found to iteratively improve the Python code as feedback to the …

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

36 extracted references · 29 canonical work pages

[1]

Wordplay Dataset repository 1pt

Andrews, M. Wordplay Dataset repository 1pt . https://github.com/mdda/cryptic-wordplay, 2024

work page 2024
[2]

and Witteveen, S

Andrews, M. and Witteveen, S. Proving that cryptic crossword clue answers are correct. In ICML 2024 Workshop on LLMs and Cognition, 2024

work page 2024
[3]

and Witteveen, S

Andrews, M. and Witteveen, S. Generating code to verify cryptic crossword reasoning. In ICLR 2025 Workshop on Deep Learning for Code, 2025

work page 2025
[4]

and Goodliffe, M

Anthony, S. and Goodliffe, M. Cracking the Cryptic (17-May-2024) . https://youtu.be/vudt7LlUX00?t=124, 2024

work page 2024
[5]

Beresford, J. R. The UK Advanced Cryptics Dictionary . Technical report, published online, 2000. https://cfajohnson.com/wordfinder/

work page 2000
[6]

Teaching large language models to self-debug

Chen, X., Lin, M., Sch \"a rli, N., and Zhou, D. Teaching large language models to self-debug. In The Twelfth International Conference on Learning Representations, 2024

work page 2024
[7]

Devious humour and painful puns: Will the cryptic crossword remain the last thing AI can't conquer? Guardian UK Crossword blog, 2024

Connor, A. Devious humour and painful puns: Will the cryptic crossword remain the last thing AI can't conquer? Guardian UK Crossword blog, 2024

work page 2024
[8]

rdeits/cryptics code repository 1pt

Deits, R. rdeits/cryptics code repository 1pt . https://github.com/rdeits/cryptics, 2015

work page 2015

Show all 36 references

[9]

CrypticCrosswords.jl code repository 1pt

Deits, R. CrypticCrosswords.jl code repository 1pt . https://github.com/rdeits/CrypticCrosswords.jl, 2022

2022
[10]

Cryptonite: A cryptic crossword benchmark for extreme ambiguity in language

Efrat, A., Shaham, U., Kilman, D., and Levy, O. Cryptonite: A cryptic crossword benchmark for extreme ambiguity in language. In Moens, M.-F., Huang, X., Specia, L., and Yih, S. W.-t. (eds.), Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing...

2021 doi
[11]

Bayesian Item Response Modeling

Fox, J.-P. Bayesian Item Response Modeling. Statistics for Social and Behavioral Sciences. Springer New York, 2010. ISBN 978-1-4419-0741-7. doi:10.1007/978-1-4419-0742-4

2010 doi
[12]

Friedlander, K. J. and Fine, P. A. The grounded expertise components approach in the novel area of cryptic crossword solving. Frontiers in Psychology, 7, 2016. ISSN 1664-1078. doi:10.3389/fpsyg.2016.00567. URL https://www.frontiersin.org/journals/psychology/articles/10.3389/fp...

2016
[13]

PAL : Program-aided language models

Gao, L., Madaan, A., Zhou, S., Alon, U., Liu, P., Yang, Y., Callan, J., and Neubig, G. PAL : Program-aided language models. In Proceedings of the 40th International Conference on Machine Learning, pp.\ 10764--10799, 2023

2023
[14]

Gemma 2: Improving open language models at a practical size, 2024

Gemma Team and Google DeepMind . Gemma 2: Improving open language models at a practical size, 2024. URL https://arxiv.org/abs/2408.00118

2024 arXiv
[15]

ToRA : A tool-integrated reasoning agent for mathematical problem solving

Gou, Z., Shao, Z., Gong, Y., Yang, Y., Huang, M., Duan, N., Chen, W., et al. ToRA : A tool-integrated reasoning agent for mathematical problem solving. In The Twelfth International Conference on Learning Representations, 2024

2024
[16]

DeepSeek-R1 : Incentivizing reasoning capability in LLMs via reinforcement learning

Guo, D., Yang, D., Zhang, H., Song, J., Zhang, R., Xu, R., Zhu, Q., Ma, S., Wang, P., Bi, X., et al. DeepSeek-R1 : Incentivizing reasoning capability in LLMs via reinforcement learning. arXiv preprint arXiv:2501.12948, 2025

2025 arXiv
[17]

J., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., Chen, W., et al

Hu, E. J., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., Chen, W., et al. Lo RA : Low-rank adaptation of large language models. In International Conference on Learning Representations, 2022

2022
[18]

Q., Welleck, S., Zhou, J

Jiang, A. Q., Welleck, S., Zhou, J. P., Lacroix, T., Liu, J., Li, W., Jamnik, M., Lample, G., and Wu, Y. Draft, sketch, and prove: Guiding formal theorem provers with informal proofs. In The Eleventh International Conference on Learning Representations, 2023

2023
[19]

J., Sutherland Robson, E., Kohli, P., de Freitas, N., Kavukcuoglu, K., and Vinyals, O

Li, Y., Choi, D., Chung, J., Kushman, N., Schrittwieser, J., Leblond, R., Eccles, T., Keeling, J., Gimeno, F., Dal Lago, A., Hubert, T., Choy, P., de Masson d’Autume, C., Babuschkin, I., Chen, X., Huang, P.-S., Welbl, J., Gowal, S., Cherepanov, A., Molloy, J., Mankowitz, D. J....

2022 doi
[20]

Macnutt, D. S. Ximenes on the art of the crossword. Methuen, 1966

1966
[21]

Advances in pre-training distributed word representations

Mikolov, T., Grave, E., Bojanowski, P., Puhrsch, C., and Joulin, A. Advances in pre-training distributed word representations. In Proceedings of the International Conference on Language Resources and Evaluation (LREC 2018), 2018

2018
[22]

Ni, A., Iyer, S., Radev, D., Stoyanov, V., Yih, W.-t., Wang, S., and Lin, X. V. Lever: Learning to verify language-to-code generation with execution. In International Conference on Machine Learning, pp.\ 26106--26128. PMLR, 2023

2023
[23]

Code generation with AlphaCodium : From prompt engineering to flow engineering

Ridnik, T., Kredo, D., and Friedman, I. Code generation with AlphaCodium : From prompt engineering to flow engineering. arXiv preprint arXiv:2401.08500, 2024

2024 arXiv
[24]

Decrypting cryptic crosswords: Semantically complex wordplay puzzles as a target for NLP

Rozner, J., Potts, C., and Mahowald, K. Decrypting cryptic crosswords: Semantically complex wordplay puzzles as a target for NLP . In Advances in Neural Information Processing Systems, volume 34, pp.\ 11409--11421, 2021

2021
[25]

Are LLMs good cryptic crossword solvers? arXiv preprint arXiv:2403.12094, 2024

Sadallah, A., Kotova, D., and Kochmar, E. Are LLMs good cryptic crossword solvers? arXiv preprint arXiv:2403.12094, 2024

2024 arXiv
[26]

Language models are crossword solvers

Saha, S., Chakraborty, S., Saha, S., and Garain, U. Language models are crossword solvers. arXiv preprint arXiv:2406.09043, 2024

2024 arXiv
[27]

Solving Olympiad geometry without human demonstrations

Trinh, T., Wu, Y., Le, Q., He, H., and Luong, T. Solving Olympiad geometry without human demonstrations. Nature, 2024. doi:10.1038/s41586-023-06747-5

2024 doi
[28]

Unsloth code repo

unsloth.ai. Unsloth code repo. https://github.com/unslothai/unsloth, 2024

2024
[29]

Automated crossword solving

Wallace, E., Tomlin, N., Xu, A., Yang, K., Pathak, E., Ginsberg, M., and Klein, D. Automated crossword solving. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp.\ 3073--3085, 2022

2022
[30]

Epic crossword battle : Expert vs

Webb, D. Epic crossword battle : Expert vs. Times cryptic puzzle \#29029. https://youtu.be/N5p4TqdjsHs, 2024

2024
[31]

V., Zhou, D., et al

Wei, J., Wang, X., Schuurmans, D., Bosma, M., Xia, F., Chi, E., Le, Q. V., Zhou, D., et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35: 0 24824--24837, 2022

2022
[32]

Cryptic crossword --- Wikipedia , the free encyclopedia

Wikipedia . Cryptic crossword --- Wikipedia , the free encyclopedia. https://en.wikipedia.org/w/index.php?title=Cryptic_crossword&oldid=1228427465, 2024. [Online; accessed 1-July-2024]

2024
[33]

Cryptic crossword - regional variation

Wikipedia contributors . Cryptic crossword - regional variation. https://en.wikipedia.org/wiki/Cryptic_crossword#Regional_variation, 2024

2024
[34]

and Woodhead, D

Williams, P. and Woodhead, D. Computer assisted analysis of cryptic crosswords. The Computer Journal, 22 0 (1): 0 67--70, 1979

1979
[35]

J., and Anandkumar, A

Yang, K., Swope, A., Gu, A., Chalamala, R., Song, P., Yu, S., Godil, S., Prenger, R. J., and Anandkumar, A. LeanDojo : Theorem proving with retrieval-augmented language models. Advances in Neural Information Processing Systems, 36: 0 21573--21612, 2023

2023
[36]

SatLM : Satisfiability-aided language models using declarative prompting

Ye, X., Chen, Q., Dillig, I., and Durrett, G. SatLM : Satisfiability-aided language models using declarative prompting. In Proceedings of NeurIPS, 2023

2023

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

Wordplay Dataset repository 1pt

Andrews, M. Wordplay Dataset repository 1pt . https://github.com/mdda/cryptic-wordplay, 2024

work page 2024

[2] [2]

and Witteveen, S

Andrews, M. and Witteveen, S. Proving that cryptic crossword clue answers are correct. In ICML 2024 Workshop on LLMs and Cognition, 2024

work page 2024

[3] [3]

and Witteveen, S

Andrews, M. and Witteveen, S. Generating code to verify cryptic crossword reasoning. In ICLR 2025 Workshop on Deep Learning for Code, 2025

work page 2025

[4] [4]

and Goodliffe, M

Anthony, S. and Goodliffe, M. Cracking the Cryptic (17-May-2024) . https://youtu.be/vudt7LlUX00?t=124, 2024

work page 2024

[5] [5]

Beresford, J. R. The UK Advanced Cryptics Dictionary . Technical report, published online, 2000. https://cfajohnson.com/wordfinder/

work page 2000

[6] [6]

Teaching large language models to self-debug

Chen, X., Lin, M., Sch \"a rli, N., and Zhou, D. Teaching large language models to self-debug. In The Twelfth International Conference on Learning Representations, 2024

work page 2024

[7] [7]

Devious humour and painful puns: Will the cryptic crossword remain the last thing AI can't conquer? Guardian UK Crossword blog, 2024

Connor, A. Devious humour and painful puns: Will the cryptic crossword remain the last thing AI can't conquer? Guardian UK Crossword blog, 2024

work page 2024

[8] [8]

rdeits/cryptics code repository 1pt

Deits, R. rdeits/cryptics code repository 1pt . https://github.com/rdeits/cryptics, 2015

work page 2015