REVIEW 4 major objections 6 minor 24 references
Primender Sequence: A Novel Mathematical Construct for Testing Symbolic Inference and AI Reasoning
T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read The Primender sequence, defined as numbers that are prime or end in a prime suffix, is proposed as an interpretable LLM symbolic-reasoning benchmark, with the paper reporting that only ChatGPT o3 inferred its rule at 5.16% error.
desk verdict A good-faith write-up of a toy sequence whose central hypothesis is a tautology and whose LLM benchmark is too uncontrolled to support any comparative conclusion. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the inclusion rule: $n$ is a Primender number exactly when $n$ is prime or one of its decimal suffixes of length 1, 2, 3, ... is prime. This rule is what makes the sequence deterministic yet hard to state from examples, and it is also what powers the structural claims—the $\Delta\le5$ bound and the $\mathrm{PE}_n-\mathrm{LP}_n=1\Rightarrow \Delta=1$ hypothesis. The evaluation machinery is a structured prompt showing the first 100 terms and asking each model to identify the rule, evaluate the hypothesis, and generate the next 100,000 terms; outputs are scored with a multiset error comparing the generated list against the true sequence.
What would settle it
Search the sequence past one million terms: finding two consecutive Primender numbers with gap at least 6, or a term with $\mathrm{PE}_n - \mathrm{LP}_n = 1$ whose predecessor is not $\mathrm{PE}_n - 1$, would refute the structural claims. For the benchmark claim, run all models under identical prompts with no human steering; if ChatGPT o3's error is not near 5.16% or another model infers the rule, the reported ranking is not reproducible.
Extended reading notes
Core claim
The central claim is that a simple hybrid rule—$n$ belongs to the Primender sequence iff $n$ is prime or $n \bmod 10^k$ is prime for some $k\ge1$—generates a sequence with stable, checkable structure and that this structure makes a good hidden-rule test for LLMs. The paper reports two structural facts: the gap $\Delta$ between consecutive Primender terms is never larger than 5, and whenever $\mathrm{PE}_n - \mathrm{LP}_n = 1$ (the $n$-th term is one more than the largest prime ≤ it), the gap from the previous term is 1. These facts are verified computationally on the first 100,000 and 1,000,000 terms. On the benchmark, the paper reports that only ChatGPT o3 inferred the rule from the first 100 terms, judged the hypothesis correctly, and generated the next 100,000 terms with a multiset error of 5.16%, while the other eight models failed at rule inference, produced high-error lists, or returned no usable list.
Load-bearing premise
The conclusions about model reasoning stand on treating each model's single chat session—with different numbers of clarifying prompts, follow-up requests, and human interventions—as a fair and comparable measure of symbolic inference, and on scoring generated lists as multisets even when a model produced fewer, extra, or differently ordered numbers.
Editorial extensions
If this is right
- The Primender sequence becomes a ready-made benchmark: any evaluator can hand a model the first 100 terms and check rule inference, hypothesis validation, and 100,000-term extrapolation against one deterministic list.
- Because the reported gap is never above 5, generated sequences can be checked cheaply: any output containing a gap of 6 or more is immediately known to be wrong, independent of the full rule.
- The result that most models validated the hypothesis but could not infer the rule suggests the benchmark isolates inductive discovery from deductive checking, a distinction central to symbolic-reasoning evaluation.
- The observation that the one successful model asked clarifying questions points to prompt designs that encourage self-questioning before sequence generation; the paper links this to Self-Ask and related techniques.
- The sequence is infinite and densely packed, so the benchmark can scale to arbitrarily long extrapolation tasks without the rule becoming obvious from the length of the sample alone.
Reading between the lines
- The paper leaves implicit that the hypothesis is a definitional consequence: if $\mathrm{PE}_n = \mathrm{LP}_n + 1$, then $\mathrm{LP}_n$ is itself prime and hence a Primender term, and because it is the integer immediately below $\mathrm{PE}_n$, the previous term in the sequence must be $\mathrm{LP}_n$, so $\Delta = 1$ necessarily.
- The gap bound $\Delta \le 5$ is also forced just by the single-digit endings 2, 3, 5, 7: among any five consecutive integers one ends in one of those digits, so a gap of 6 would be impossible even without considering longer prime suffixes or primality.
- A sharper benchmark would exclude single-digit-qualified numbers (require the qualifying suffix to have length at least 2, or require the whole number to be prime), forcing models to attend to the modular-suffix structure rather than the common last-digit residue.
- A testable extension: give every model a mandatory self-ask turn before generating terms; if the paper's explanatory-questioning explanation is right, models that previously failed rule inference should show substantially lower error.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces the Primender sequence, defined as the integers that are prime or have at least one prime suffix (of any length), and proposes it as a benchmark for evaluating symbolic reasoning in large language models. The authors state that the maximum difference between consecutive Primender numbers is 5, and they formulate a hypothesis that whenever PEn - LPn = 1 (where PEn is a Primender number and LPn is the largest prime at most PEn), the gap from the previous Primender number is also 1. They then prompt nine LLMs to infer the rule, evaluate the hypothesis, and generate the next 100,000 terms, reporting that only ChatGPT o3 inferred the rule and achieved a 5.16% error rate under a multiset-based scoring metric. The paper also provides computational verification up to 1,000,000 terms and makes code and data available in a GitHub repository and an OEIS entry.
Significance. If the mathematical claims were rigorously proved and the LLM evaluation were a controlled, apples-to-apples comparison, the Primender sequence could serve as a simple, interpretable symbolic benchmark. The paper does provide reproducible computational artifacts (Python code, generated data, and an OEIS registration), and the central ``max delta 5'' statement is in fact true and admits an elementary proof. However, the manuscript as written has load-bearing flaws: the proof of the main mathematical claim is invalid and outsourced to a non-archival blog, the central hypothesis is a definitional tautology rather than an empirical discovery, and the LLM comparison is neither a controlled experiment nor scored with a metric appropriate for sequence generation. These issues undermine the paper's core claims as they stand.
major comments (4)
- [Section IV and Section VI] The hypothesis PEn - LPn = 1 implies Δ = 1 is not an empirical pattern but a direct consequence of the sequence definition. If PEn - LPn = 1, then LPn = PEn - 1, and since LPn is prime, LPn is itself a Primender number. Hence the previous Primender before PEn is exactly LPn, so Δ = 1. The authors present this as a discovered conjecture and ask LLMs to evaluate it, but the problem is circular and does not test meaningful symbolic inference. This should be stated and proved as a one-line lemma, not treated as the centerpiece of a benchmark.
- [Section III.C] The proof that the maximum delta is 5 is invalid as written. Invoking Bertrand's postulate does not show that any six consecutive integers contain a prime, and the disjunction of conditions is asserted without justification. The proof is then referred to the author's own blog [14], which is not an archival source. The claim itself is true and has a simple proof: among any six consecutive integers, at least one has last digit 2, 3, 5, or 7 and therefore qualifies by the unit-digit rule; the bound is attained, for example, by the gap from 47 to 52. The authors should replace the flawed argument with this elementary proof.
- [Section VI and Fig. 8] The LLM evaluation is not controlled: models received different numbers of prompting turns, some received answers to clarifying questions (ChatGPT o3), one was run with a special 'Think For Longer' feature (ChatGPT 4), some outputs were produced by Python programs returned by the model rather than directly by the model (DeepSeek R1), one model's copied list contained only about 2000 numbers (Gemini 2.5 Pro), and several required repeated prompts for downloadable output. These are not equivalent tasks, so the reported error rates and the conclusion that only ChatGPT o3 inferred the rule cannot be supported. A benchmark claim requires a fixed protocol applied identically to all models, with all interventions and follow-up prompts documented.
- [Section VI, 'Ignore Order Compare Content'] The scoring metric is inappropriate for the stated task. The task asks models to generate the next 100,000 terms in order, but the Python comparison uses collections.Counter and treats the outputs as multisets, giving full credit for correct numbers regardless of position. A model that outputs all correct terms in a permuted order would receive a 0% error rate under this metric, while a model that outputs only a short prefix is penalized in a way that conflates length, ordering, and rule inference. The reported 5.16% error rate for ChatGPT o3 therefore does not demonstrate correct sequence inference, and the error-rate column in Fig. 8 is not a measure of sequence-generation quality.
minor comments (6)
- [Section V] The prompt shown to the models already reveals that the sequence is called 'Primender' and hints that the rule involves ending digits; this may artificially guide rule inference and should be acknowledged as a possible confound.
- [Section VI, Fig. 8] The 'Rule Inference Accuracy' column is a binary yes/no with no defined threshold or validation procedure; the basis for assigning 'No' to eight models and 'Yes' to ChatGPT o3 is not stated.
- [Section III.D] The sentence 'We encourage scholars to cite and explore the Primender series by the author of this research' is promotional and inappropriate in a research paper; it should be removed.
- [References] References [21] and [22] are identical arXiv identifiers (arXiv:2210.03350), and reference [24] appears to have an incomplete or erroneous author list; these citation problems should be corrected.
- [General] There are numerous typos and ungrammatical sentences (e.g., 'hypotheisis', 'gerated', 'it didn't evaluated', 'the pattern found was incorrect'), and large code blocks interrupt the narrative; the code should be moved to an appendix or repository.
- [Section IV.B] The claim that PEn - LPn can take any unit digit except 7 is asserted without proof and only verified empirically; a rigorous argument or a clear statement that it is a conjecture is needed.
Circularity Check
The paper's key hypothesis — if PEn − LPn = 1, then Δ = 1 — reduces by construction to the Primender definition (LPn is prime, hence a member, hence the preceding term), yet is presented as a discovered micro-pattern and becomes the benchmark's hypothesis-evaluation axis; the headline LLM comparison is additionally confounded by unequal prompting and an order-insensitive error metric.
-
self definitional
[Section IV 'Designing a Hypothesis'; abstract key-hypothesis statement; prompt task 3 in Section V]
"By visually inspecting and analyzing this structured data, we observed a recurring micro-pattern: whenever a Primender number was exactly one more than the largest prime less than or equal to it (i.e., PEn − LPn = 1), the delta from the previous Primender was also 1. This led us to formulate the following hypothesis: If PEn − LPn = 1, then Δ = 1."
Under the paper's own definitions, PEn − LPn = 1 means PEn = LPn + 1 with LPn prime. Section III's inclusion rule ('x is a prime number, or...') makes every prime a Primender, so LPn is a Primender below PEn with no integer strictly between them; the preceding Primender term is exactly LPn, hence Δ = 1. The implication is forced by the sequence's definition in one line; it is not an empirical micro-pattern. The paper reports visually observing and computationally 'verifying' this tautology over 1,000 and 100,000 terms, and prompt task 3 (with task 2 already defining LPn as the nearest prime) asks LLMs to evaluate it as an open hypothesis. The hypothesis-evaluation axis of Fig. 8 and the claimed hypothesis-testing contribution thereby reduce to the defining inclusion rule itself.
full rationale
One genuine circular step. The 'key hypothesis' (abstract; Section IV; prompt task 3) is a one-line consequence of the Primender membership rule: PEn − LPn = 1 forces LPn = PEn − 1 prime, and since Section III admits every prime ('x is a prime number, or...'), LPn is itself the immediately preceding Primender term, so Δ = 1. The framing — observing a 'recurring micro-pattern' and 'verifying this hypothesis computationally' for 1,000 and 100,000 terms — presents a definitional identity as an empirical discovery, and the LLM task of evaluating it is a tautology check. Section III.D's 'Every time the delta is 4, the second number in the pair of Primender indeed ends with the digit 1... confirmed with the first 100,000' has the same character (a digit-case consequence of the last-digit rule) though it is a side observation, not a benchmark axis. The rule-inference and sequence-generation axes (tasks 1 and 4) are independent, so the circularity is partial. The benchmark comparison is confounded, and I weigh these as correctness risks rather than circular steps per the rules. Fig. 8 compares unequal tasks: the paper's own notes state ChatGPT o3 'clarified 3 points and we provided relevant answers' (and its rule was only 'till last two digits to be prime,' not the full rule, despite the 'Yes' credit), LLaMA 'had to provide multiple inputs through prompts,' Gemini 2.5 Pro's list 'when copied even that had only around 2000 numbers' yet is scored 99.46%, DeepSeek's scored list was 'Generated list using the program it returned,' and the paper concedes 'human intervention was necessary' in several cases. The error metric is explicitly order-insensitive ('Ignore Order Compare Content... when the position of elements doesn't matter') for a task asking to 'Generate the next 100,000 terms' of an ordered sequence whose own Delta series is order-defined, so a permuted or partial output is not penalized as a sequence failure. The self-citations (repository [10], blog proof [14], OEIS A384735 [15], checker [16]) are numerous but not load-bearing: the maximum-delta claim follows directly from the last-digit inclusion rule (any 5 consecutive integers contain one ending in 2, 3, 5, or 7) and is code-verified in-paper, so the blog deferral does not force the result. Overall: partial circularity driven by the definitional hypothesis, score 6.
Assumptions & free parameters
assumptions (3)
- ad hoc to paper The validity of the claimed proof that maximum delta is 5, as presented in the author's blog post [14], is assumed.
- domain assumption The multiset (order-ignoring) comparison of generated lists against the true sequence is a valid measure of sequence generation quality.
- domain assumption The single-session, variably prompted interactions with each LLM are representative of the model's symbolic reasoning ability.
invented entities (1)
-
Primender sequence
independent evidence
Cite this review
Pith. "Pith review of Primender Sequence: A Novel Mathematical Construct for Testing Symbolic Inference and AI Reasoning." pith.science (2026). https://pith.science/paper/Z4LCA4JK
@misc{pith2026250610585,
author = {Pith},
title = {Pith review of: Primender Sequence: A Novel Mathematical Construct for Testing Symbolic Inference and AI Reasoning},
year = {2026},
howpublished = {\url{https://pith.science/paper/Z4LCA4JK}},
note = {Machine review of arXiv:2506.10585}
}
read the original abstract
This paper introduces the Primender sequence, a novel integer sequence defined by a hybrid rule that combines classical primality with modular digit-based conditions. Specifically, a number n is included in the sequence if it is prime or ends with a prime number of unit digit or any length. In other words, numbers which are primes or have at least one prime suffix. The resulting sequence exhibits a deterministic yet non-trivial structure, blending number-theoretic properties with symbolic patterning. We propose the Primender sequence as a benchmark for evaluating the symbolic reasoning capabilities of Large Language Models (LLMs). The study is motivated by the need for interpretable, rule-based testbeds that can assess an LLM's ability to infer hidden rules, validate mathematical hypotheses, and generalize symbolic logic at scale. A key hypothesis explored is: Whenever a number in the Primender sequence is exactly one more than the largest prime less than or equal to it, the difference between it and the previous number in the sequence is also 1. We design a structured prompt and evaluation framework to test this hypothesis across multiple state-of-the-art LLMs, including ChatGPT, Copilot, DeepSeek, Gemini, Grok, and LLaMA. The models are tasked with identifying the underlying rule, validating the hypothesis, and generating the next 100,000 terms of the sequence. Comparative metrics such as rule inference accuracy, hypothesis evaluation, sequence validity, and symbolic explanation quality are used to assess model performance. This work contributes a novel mathematical construct and a reproducible methodology for benchmarking LLMs in symbolic reasoning, hypothesis testing, and scalable pattern generalization - bridging the domains of number theory, artificial intelligence, and software engineering.
Reference graph
Works this paper leans on
-
[14]
Primender Sequence: Numbers that Are Prime or End in Prime,
Mohd Anwar Jamal Faiz, “Primender Sequence: Numbers that Are Prime or End in Prime,” *W3LC Blog*, Jun. 2025. [Online]. Available: https://www.w3lc.com/2025/06/primender-sequence-numbers-that- are.html
work page 2025
-
[1]
Ueber die Anzahl der Primzahlen unter einer gegebenen Grösse,
B. Riemann, “Ueber die Anzahl der Primzahlen unter einer gegebenen Grösse,” *Monatsberichte der Berliner Akademie*, pp. 671–680, 1859. [English Trans. Available: https://www.maths.tcd.ie/pub/HistMath/People/Riemann/Zeta/]
-
[2]
Euclid, *Elements*, Book IX, Prop. 20. ca. 300 BCE. [Online]. Available: https://mathcs.clarku.edu/~djoyce/elements/bookIX/propIX20.html
-
[3]
A method for obtaining digital signatures and public-key cryptosystems,
R. L. Rivest, A. Shamir, and L. Adleman, “A method for obtaining digital signatures and public-key cryptosystems,” *Communications of the ACM*, vol. 21, no. 2, pp. 120–126, Feb. 1978
work page 1978
-
[4]
Pretrained language models are symbolic mathematics solvers too!
K. Noorbakhsh, M. Sulaiman, M. Sharifi, K. Roy, and P. Jamshidi, “Pretrained language models are symbolic mathematics solvers too!”, *arXiv*, Oct. 2021
work page 2021
-
[5]
The Role of Benchmarking in Symbolic Computation: (Position Paper),
J. Davenport, “The Role of Benchmarking in Symbolic Computation: (Position Paper),” in *Proc. 20th Int. Symp. Symbolic and Numeric Algorithms for Scientific Computing (SYNASC)*, 2018
work page 2018
-
[6]
Tree of Thoughts: Deliberate Problem Solving with Large Language Models,
S. Yao, D. Yu, J. Zhao, I. Shafran, T. Griffiths, Y. Cao, and K. Narasimhan, “Tree of Thoughts: Deliberate Problem Solving with Large Language Models,” in *Advances in Neural Information Processing Systems*, vol. 36, 2024
work page 2024
-
[7]
C. F. Gauss, *Disquisitiones Arithmeticae*, Leipzig: Fleischer, 1801. [English translation by A. A. Clarke, Yale University Press, 1966]
work page 1966
Show all 24 references
-
[8]
E. C. Titchmarsh, *The Theory of the Riemann Zeta -Function*, 2nd ed., revised by D. R. Heath -Brown. Oxford, UK: Oxford University Press, 1986. [Originally published 1930]
1986
-
[9]
Representation of an odd number as a sum of three primes,
I. M. Vinogradov, “Representation of an odd number as a sum of three primes,” *Doklady Akademii Nauk SSSR*, vol. 15, no. 6, pp. 291 – 294, 1937
1937
-
[10]
Github repository for Primender,
Mohd Anwar Jamal Faiz, “Github repository for Primender,” GitHub,
-
[11]
Bounded gaps between primes,
Y. Zhang, “Bounded gaps between primes,” *Annals of Mathematics*, vol. 179, no. 3, pp. 1121 –1174, 2014. doi: 10.4007/annals.2014.179.3.6
2014 doi
-
[12]
J. W. Tukey, *Exploratory Data Analysis*. Reading, MA: Addison - Wesley, 1977. A foundational book introducing scatter plots and graphical methods to analyze variation and distribution in data sequences
1977
-
[13]
A proof of Bertrand’s postulate,
S. Ramanujan, “A proof of Bertrand’s postulate,” *Journal of the Indian Mathematical Society*, vol. 11, pp. 181–182, 1919
1919
-
[15]
A384735: Primender Sequence: Numbers that are prime or end in a prime number (of any length),
Mohd Anwar Jamal Faiz, “A384735: Primender Sequence: Numbers that are prime or end in a prime number (of any length),” *The On-Line Encyclopedia of Integer Sequences*, published May 2024. [Online]. Available: https://oeis.org/A384735
2024
-
[16]
The Primender Playground : A Toolkit for Number-Theory Enthusiasts ,
Mohd Anwar Jamal Faiz, “The Primender Playground : A Toolkit for Number-Theory Enthusiasts ,” [Online]. Available: https://tools.anwarjamal.com/primender
-
[17]
Improving language understanding by generative pre -training,
A. Radford, K. Narasimhan, T. Salimans, and I. Sutskever, “Improving language understanding by generative pre -training,” OpenAI, San Francisco, CA, USA, Tech. Rep., Jun. 2018. [Online]. Available: https://cdn.openai.com/research-covers/language- unsupervised/language_understa...
2018
-
[18]
Attention is all you need,
A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” in *Advances in Neural Information Processing Systems*, vol. 30, pp. 5998–6008, 2017
2017
-
[19]
Comparative Analysis of Top LLMs: GPT-4, Gemini, Claude and More,
A. Jain, “Comparative Analysis of Top LLMs: GPT-4, Gemini, Claude and More,” *Baeldung AI*, Apr. 2024. [Online]. Available: https://baeldung.com/llm-benchmark
2024
-
[20]
Humanity’s Last Exam: Evaluating LLM Reasoning,
Scale AI, “Humanity’s Last Exam: Evaluating LLM Reasoning,” *Wikipedia*, 2024. [Online]. Available: https://en.wikipedia.org/wiki/Humanity%27s_Last_Exam
2024
-
[22]
Self - Ask: A Simple Prompting Technique for Improving Reasoning in Language Models,
D. Press, M. Barak, L. Du, X. Liu, R. L. West, and N. Shazeer, “Self - Ask: A Simple Prompting Technique for Improving Reasoning in Language Models,” arXiv preprint arXiv:2210.03350, Oct. 2022. [Online]. Available: https://arxiv.org/abs/2210.03350
2022 arXiv
-
[23]
Improving intermediate reasoning in zero -shot chain-of-thought for large language models via Filter -Supervisor Self-Correction,
M. Nicolae et al., “Improving intermediate reasoning in zero -shot chain-of-thought for large language models via Filter -Supervisor Self-Correction,” Neurocomputing, vol. 610, pp. 41 –55, Jan 2025. doi: 10.1016/j.neucom.2024.02.123
2025 doi
-
[24]
Self - Refine: Iterative Refinement with Self -Feedback,
Madaan, B. Nye, X. Li, R. B. Zellers, A. Farhadi, and Y. Choi, “Self - Refine: Iterative Refinement with Self -Feedback,” arXiv preprint arXiv:2303.17651, Mar. 2023. [Online]. Available: https://arxiv.org/abs/2303.17651
2023 arXiv
-
[2025]
Available: https://github.com/Anwar -Faiz/primender
[Online]. Available: https://github.com/Anwar -Faiz/primender. [Accessed: Jun. 09, 2025]
2025
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.