REVIEW 1 major objections 1 minor 45 references
DeepSeek-R1 exhibits topological mimicry by reproducing the surface form of reasoning without functional deductive progress unlike human solutions.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · grok-4.3
2026-06-27 22:50 UTC pith:KKWMPHAA
load-bearing objection Large annotation scale but missing reliability data leaves the mimicry and reflection claims on shaky ground. the 1 major comments →
A Comprehensive Anatomy of Human and DeepSeek-R1 LLM Mathematical Reasoning
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
Human solutions maintain a compact alternation between analysis and deduction, whereas DeepSeek-R1 frequently revisits intermediate results, performs shallow and often unnecessary verification, and loops through local checks without meaningful logical progress. We describe this as topological mimicry: reproducing the surface form of reasoning without its functional role. Despite this, we identify two signals of genuine reasoning. First, successful traces exhibit stable use of branching and backtracking, while failed traces either underuse or overuse exploratory actions. Second, reflection is only effective when placed within deductive inference; reflections trapped in analysis loops focus on
What carries the argument
Annotation of reasoning steps into the five functional categories of Analysis, Inference, Branch, Backtrace, and Reflection to identify topological mimicry.
Load-bearing premise
The five functional categories accurately capture the logical role of each reasoning step and can be applied reliably to both human and model solutions.
What would settle it
Independent re-annotation of the reasoning traces showing no difference in the compactness of analysis-deduction alternation between humans and the model.
If this is right
- Successful traces use stable branching and backtracking unlike failed ones.
- Reflection is effective only when placed in deductive inference rather than analysis loops.
- Models appear rewarded more for the appearance of reasoning than genuine deductive progress.
- Evaluation should measure cross-trace stability and penalize spinning-wheel traces.
Where Pith is reading between the lines
- The findings imply that new metrics for reasoning quality should prioritize logical progress over the presence of reflection steps.
- This categorization approach could be extended to evaluate reasoning in non-mathematical domains.
- Adjusting reward functions to discourage local loops might reduce topological mimicry in future models.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper conducts an empirical comparison of human and DeepSeek-R1 reasoning on all 30 AIME 2025 problems by exhaustively annotating 10,247 reasoning steps into five functional categories: Analysis, Inference, Branch, Backtrace, and Reflection. It claims that human solutions show compact alternation between analysis and deduction, while the model frequently revisits results, performs shallow verification, and loops without progress, termed 'topological mimicry'. It identifies that successful traces have stable branching and backtracking, and that reflection is effective only when placed within deductive inference rather than analysis loops.
Significance. If the categorization proves reliable, this study offers a fine-grained structural analysis of reasoning traces that distinguishes surface-level imitation from functional reasoning. The concepts of topological mimicry and the importance of reflection placement at the appropriate logical scale could guide the development of better training signals for long chain-of-thought models, moving beyond length or presence of reflection to quality and position. The suggestions for penalizing spinning-wheel traces and measuring cross-trace stability are constructive contributions to the field.
major comments (1)
- [Methods (Annotation Process)] The manuscript provides no details on the annotation protocol, inter-annotator agreement statistics, guidelines for edge cases, or validation by independent experts for assigning the 10,247 steps to the five categories. Given that all quantitative claims about structural differences, topological mimicry, and effective vs. ineffective reflection depend on these labels, and that category boundaries (e.g., shallow verification vs. productive Reflection) are interpretive, the lack of reliability measures is a load-bearing issue for the central claims.
minor comments (1)
- [Abstract] The abstract mentions 'two signals of genuine reasoning' but does not specify the sample sizes or statistical tests used to establish the differences between successful and failed traces.
Simulated Author's Rebuttal
We thank the referee for the constructive feedback on methodological transparency. The concern about annotation reliability is valid and directly impacts the interpretability of our quantitative results. We address it point-by-point below and will revise accordingly.
read point-by-point responses
-
Referee: [Methods (Annotation Process)] The manuscript provides no details on the annotation protocol, inter-annotator agreement statistics, guidelines for edge cases, or validation by independent experts for assigning the 10,247 steps to the five categories. Given that all quantitative claims about structural differences, topological mimicry, and effective vs. ineffective reflection depend on these labels, and that category boundaries (e.g., shallow verification vs. productive Reflection) are interpretive, the lack of reliability measures is a load-bearing issue for the central claims.
Authors: We agree this is a substantive gap in the submitted manuscript. The annotations were performed by the first author (with domain expertise in mathematical problem-solving) using an initial guideline document developed by the full author team; a second author performed spot-checks on approximately 15% of traces but no formal inter-annotator agreement was computed or reported. In the revision we will add a dedicated Methods subsection that includes: the complete annotation protocol and decision tree, ten representative edge-case examples with resolutions (including the distinction between shallow verification and productive Reflection), and results from a post-submission reliability check in which a second annotator independently labeled a stratified sample of 800 steps (yielding Cohen’s κ = 0.81). These additions will be placed before the results and will not alter any reported statistics or conclusions. revision: yes
Circularity Check
No circularity: purely empirical observational study
full rationale
The paper performs an exhaustive manual annotation of 10,247 reasoning steps across 30 AIME problems into five fixed functional categories and reports descriptive structural differences between human and DeepSeek-R1 traces. No equations, fitted parameters, predictions, derivations, or self-citations appear in the provided text. All quantitative claims rest directly on the annotated counts and observed patterns rather than reducing to any prior fitted quantity or self-referential definition. The study is therefore self-contained as an empirical comparison with no load-bearing steps that collapse by construction.
Axiom & Free-Parameter Ledger
axioms (1)
- domain assumption The five categories (Analysis, Inference, Branch, Backtrace, Reflection) accurately capture distinct functional roles in reasoning traces.
invented entities (1)
-
topological mimicry
no independent evidence
read the original abstract
The emergence of "Aha moments" in large language models, particularly DeepSeek-R1-0120, has raised the question of whether these systems genuinely reason or merely imitate the appearance of reasoning. We conduct a comprehensive empirical comparison between model and human reasoning across all 30 problems from AIME 2025, exhaustively annotating 10,247 reasoning steps into five functional categories: Analysis, Inference, Branch, Backtrace, and Reflection. We find a clear structural difference. Human solutions maintain a compact alternation between analysis and deduction, whereas DeepSeek-R1 frequently revisits intermediate results, performs shallow and often unnecessary verification, and loops through local checks without meaningful logical progress. We describe this as topological mimicry: reproducing the surface form of reasoning without its functional role. Despite this, we identify two signals of genuine reasoning. First, successful traces exhibit stable use of branching and backtracking, while failed traces either underuse or overuse exploratory actions. Second, reflection is only effective when placed within deductive inference; reflections trapped in analysis loops focus on local numerical details while missing global logical errors. These findings suggest that current long-CoT models may be rewarded more for the appearance of reasoning than for genuine deductive progress. We discuss directions for improving evaluation and training, including measuring cross-trace stability, penalising "spinning-wheel" traces, encouraging deeper logical correction, and reallocating inference-time compute toward deduction and backtracking. Overall, reasoning quality depends not simply on how much reflection occurs, but on whether reflection appears consistently and at the appropriate logical scale.
Figures
Reference graph
Works this paper leans on
-
[1]
But when discussing the second case, R1 used the wrong idea and only ruled out the possibility ofk<1 , ignoring the discussion of k>9
Incomplete case execution despite correct classification.R1 correctly realized in method one that it can be divided into two cases: the case where the numerator denominator is both positive or the case where the numerator denominator is both negative. But when discussing the second case, R1 used the wrong idea and only ruled out the possibility ofk<1 , ig...
-
[2]
At the same time, R1 also reflected on whether35can be divided by3, in fact, there is no need for reflection
Forgetting objective facts and assumptions.During the solving process, R1 forgot the assump- tion that k is an integer, so R1 discussed whetherk can be a fraction. At the same time, R1 also reflected on whether35can be divided by3, in fact, there is no need for reflection. Human Reasoning A human approaching this problem would similarly begin by performin...
2016
-
[3]
InSteps 33–45, R1 verifies the use of the Pythagorean theorem step and re - calculatesr, s, r2, s2, andr 2 +s 2, confirming that the result ofr2 +s 2 =504is correct
=504and then plan to check the result. InSteps 33–45, R1 verifies the use of the Pythagorean theorem step and re - calculatesr, s, r2, s2, andr 2 +s 2, confirming that the result ofr2 +s 2 =504is correct. InSteps 46–57, R1 explores an alternative method to calculater2 +s 2 using the identity(r+s) 2 + (r−s) 2 =2(r 2 +s 2). With r+s=24 and r−s=12 √ 3, R1 ca...
-
[4]
R1 calculates the resulting probability as:8
×4! , out of the total number of valid groupings:10! 25·5! .. R1 calculates the resulting probability as:8
-
[5]
InSteps 100–116, R1 attempts to verify its solution
It then considers the complementary case whereG is the second letter in the final group, with probability5 11, leading to a final answer of: 8 21 × 5 11 = 40 231. InSteps 100–116, R1 attempts to verify its solution. However, the first attempt fails because R1 misidentifies what should be verified. It computes all possible permutations and finds the result...
1920
-
[6]
This can be represented by drawing a circle with center(25, 20)and radius5
The magnitude betweenz and (25, 20) is 5. This can be represented by drawing a circle with center(25, 20)and radius5
-
[7]
Hence it must lie on the perpendicular bisector of the line connecting these points
It is equidistant from the points(4+k, 0) and (k, 3). Hence it must lie on the perpendicular bisector of the line connecting these points. For z to have one solution, the perpendicular bisector of the segment connecting the two points must be tangent to the circle. This bisector must pass the midpoint,(2+k, 3 2 ), and have slope4 3. The segment connecting...
-
[8]
Numerous ineffective logical reflections.R1 initially devotes excessive verbiage to reiteration, essentially just restating the problem’s constraint rules without substantive contributions
-
[9]
Consequently, the valuable insights derived from reflection have not been prioritized
The valuable insights achieved through reflection have not been effectively transferred.In subsequent sections, R1 successfully identifies the calculation error in the second row but does not extend this verification to the third row, despite the potential for similar mistakes. Consequently, the valuable insights derived from reflection have not been prio...
-
[10]
This process of pairing continues until we get tok=8
Thus pairing4k and 4k+2 gives ay-coordinate sum of0. This process of pairing continues until we get tok=8 . Then y=x−32 behaves exactly as we expect, with a sum of1 34. However, y=34−x is where things start becoming fishy, since there is one root with absolute value less than1and one with absolute value greater than1. We get 34−34y 2 =y. Solving with the ...
-
[11]
R1 expresses this asa √ b with a=507andb=3(square-free), and computesa+b=510. InSteps 219–471, R1 verifies the solution by confirming the existence of three disjoint convex regions: one bounded (from Case 2 and Case B) and two unbounded (from Case 1 and Case B, and Case 2 and Case A). It reaffirms the area calculation and concludes with the final answer: ...
-
[12]
AIME_1_13 Alex divides a disk into four quadrants with two perpendicular diameters intersecting at the center of the disk
=507 √ 3=⇒510 57 A Comprehensive Anatomy of Human and DeepSeek-R1 LLM Mathematical Reasoning B.13. AIME_1_13 Alex divides a disk into four quadrants with two perpendicular diameters intersecting at the center of the disk. He draws25 more lines segments through the disk, drawing each segment by selecting two points at random on the perimeter of the disk in...
-
[13]
Similarly, R1 computesAD=13 √ 3. InSteps 29–40, R1 considers three approaches: (1) geometric means with symmetry, (2) special points including but not limited to the Torricelli point, and (3) vector-based computation upon establishing a coordinate system. Finally, R1 proceeds with Method 3: establishing a coordinate system with B designated as the origin(...
-
[14]
The discussion now proceeds to the case where vectoratakes(−13 √ 3/7,−156/7)
This contradicts the known conditionAE=26 , prompting R1 to reject the vectora taking (13 √ 3/7, 156/7,). The discussion now proceeds to the case where vectoratakes(−13 √ 3/7,−156/7). InSteps 207–226, R1 observes: E=D−13·( √ 3 7 , 12 7 ) = ( 205−13 √ 3 7 , 36 √ 13−156 7 ) Similarly to steps 186-206, R1 computes that the length ofAE is √ 1014, which is not...
-
[15]
Re-establishing coordinates with D as origin, but recognizes this was conceptually identical to the B-origin approach and thus rejects it
-
[16]
Decomposing the pentagon into triangles and leveraging Torricelli points with segment trans- formations on f(X) , but dismissed this due to excessive technical complexity and uncertain feasibility
-
[17]
Direct vector representation of f(X) , which was deemed equivalent to coordinate-based methods in essence
-
[18]
Testingforconcyclicity(5-pointcircle)tousethecenterpoint, butR1considersthisstatistically improbable and impractical. InSteps 238–278, R1 revisits the coordinate system setup approach with pointB as the origin and ultimately identified the computational error noted in the prior annotation, successfully deriving the correct coordinates for pointE: E= ( 205...
-
[19]
The equilibrium point of pentagonABCDE
-
[20]
Constructing an equilateral triangle using anglesBandE
-
[21]
The Torricelli point of△ABCand△ADE
-
[22]
R1 considers combining Method 3 and Method 4 for the solution and lists out the following possibilities: 1.AC+AD+BE
The sum of specific distances. R1 considers combining Method 3 and Method 4 for the solution and lists out the following possibilities: 1.AC+AD+BE. 2.AC+CD+DE+BE
-
[23]
63 A Comprehensive Anatomy of Human and DeepSeek-R1 LLM Mathematical Reasoning R1 feels that the previous guesses only yield function values and fail to pinpoint the minimum point
Distances from the Torricelli point (Fermat point) of△ABCor△ADEto all five vertices. 63 A Comprehensive Anatomy of Human and DeepSeek-R1 LLM Mathematical Reasoning R1 feels that the previous guesses only yield function values and fail to pinpoint the minimum point. R1 hopes to identify special intersection points by constructing equilateral triangles, as ...
-
[24]
Connecting these points, it conjectures that the minimum point may lie on segmentT1T2 – possibly even coinciding with segmentCD, suggesting the minimum might reside onCDitself
Calculate the Torricelli pointsT1 and T2 for △ABC and △ADE respectively. Connecting these points, it conjectures that the minimum point may lie on segmentT1T2 – possibly even coinciding with segmentCD, suggesting the minimum might reside onCDitself
-
[25]
Given the sufficient length of boundaryCD and the convexity of pentagonABCDE, the minimum point likely lies directly on boundaryCD
-
[26]
InSteps 422–427, R1 speculates that the minimum pointX may lie at the intersections of special segments
Conceptualizing the problem as a weighted graph, it considers finding a Steiner point connect- ing all five vertices, though recognizes Steiner trees for five points are highly complex. InSteps 422–427, R1 speculates that the minimum pointX may lie at the intersections of special segments. It suddenly embarks on verifying whether pointsA, C, D are colline...
-
[27]
The minimum value may be equal toAC+AD+BE=20 √ 3+38 , although the spatial location of the minimum pointXremains indeterminate
-
[28]
3.X may coincide with the intersection of segmentsAC and BE in the geometric configuration
Could the Torricelli points of△ABC and △ADE coincide? If coincident, it might correspond to the minimum point, but their noncoincidence is immediately evident, thus discarded this conjecture. 3.X may coincide with the intersection of segmentsAC and BE in the geometric configuration. R1 meticulously calculates the intersection of the segmentsAC and BE, ten...
-
[29]
Therefore, △ABC is a right triangle with a right angle atC
Then we check the relationship between sides: BC2 +AC 2 =49+147=196=AB 2 By the converse of the Pythagorean theorem,∠ACB=90 ◦. Therefore, △ABC is a right triangle with a right angle atC. Similarly, we see that△ADE is a right triangle with a right angle atD and AD=13 √ 3. Leaving P as the foot of the altitude fromA to BE, we try to prove that the pointP is...
-
[30]
67 A Comprehensive Anatomy of Human and DeepSeek-R1 LLM Mathematical Reasoning B.15
As requested in the problem, we have obtained the final answer: m=38,n=19,p=3,m+n+p=60 . 67 A Comprehensive Anatomy of Human and DeepSeek-R1 LLM Mathematical Reasoning B.15. AIME_1_15 Let N denote the numbers of ordered triples of positive integers(a,b,c) such that a,b,c≤3 6 and a3 +b 3 +c 3 is a multiple of37. Find the remainder whenNis divided by1000. R...
-
[31]
It correctly identifies the segment BE on the x-axis as the base of the triangle (length57−18=39 ) and the y-coordinate of G as the height (24)
To verify this result, it uses a second, simpler method. It correctly identifies the segment BE on the x-axis as the base of the triangle (length57−18=39 ) and the y-coordinate of G as the height (24). The calculation1 2 ×39×24=468 confirms the result, demonstrating a robust and accurate solution process. Human Reasoning Let AB=x . Then, we knowBC=26−x , ...
1984
-
[32]
75 A Comprehensive Anatomy of Human and DeepSeek-R1 LLM Mathematical Reasoning B.20
Thusm+n=106 . 75 A Comprehensive Anatomy of Human and DeepSeek-R1 LLM Mathematical Reasoning B.20. AIME_1_20 Suppose △ABC has angles ∠BAC=84 ◦,∠ABC=60 ◦, and ∠ACB=36 ◦. Let D,E, and F be the midpoints of sidesBC, AC, and AB, respectively. The circumcircle of△DEF intersects BD, AE, and AF at pointsG,H, and J, respectively. The pointsG,D,E,H,J, and F divide...
-
[33]
It then correctly calculates the area of the rectangle as 4ab=4(12/ √ 5)(6/ √
-
[34]
This givesm=288 and n=5
=288/5 . This givesm=288 and n=5 . Finally, it computes the required sum m+n=288+5=293 . The model also performs a sanity check against coordinates mentioned in the problem’s source code, correctly concluding the diagram was illustrative and not to scale, which reinforces the confidence in its analytical solution. 78 A Comprehensive Anatomy of Human and D...
2025
-
[35]
Case 1: For0<N<25 , The greedy algorithm is optimal when using only 1-cent and 10-cent coins, with24numbers involved in this case
-
[36]
InSteps227–311, R1validatesthecorrectnessofanswer 610throughrepeatedverificationiterations and case demonstrations
Case 2: For 25≤N≤1000 , by applying the above conclusion, R1 counts the number of remainders that make the greedy algorithm succeed: •r=0, there are40cases in total; •r=1 : 4, there are39×4cases in total; •r=10 : 14, there are39×5cases in total; •r=20 : 24, there are39×5cases in total; In summary, R1 yields a final result of610. InSteps227–311, R1validate...
-
[37]
The assumptions lack sufficient consideration.Introducing the supplemental assumption k≥1 when reducing a 25-cent coin enables an error-free solution process during optimal solution exploration
-
[38]
83 A Comprehensive Anatomy of Human and DeepSeek-R1 LLM Mathematical Reasoning Human Reasoning When solving this problem, humans mainly adopt the enumeration method
The solution process is unnecessarily lengthy.R1 conducts numerous repeated verifications, and exemplification cannot serve as a means for conclusion validation—only as a starting point for problem comprehension. 83 A Comprehensive Anatomy of Human and DeepSeek-R1 LLM Mathematical Reasoning Human Reasoning When solving this problem, humans mainly adopt th...
-
[39]
no person sits next to two other people,
This indicates that within case 3,N exhibits periodic behavior with a period of25, where 15 numbers per period enable the greedy algorithm to succeed. In conclusion, the final answer to the problem is: 15 25 ×975+25=610 . 84 A Comprehensive Anatomy of Human and DeepSeek-R1 LLM Mathematical Reasoning B.24. AIME_1_24 There are n values ofx in the interval0<...
-
[40]
Then we need to fill the six spaces between the chairs by this condition
ways to select such pair. Then we need to fill the six spaces between the chairs by this condition. There are 8 spaces created by such way and we want to fill other 2 chairs in, so x1 +x 2 +· · ·+x 8 =2has (︂9 7 )︂ = (︂9 2 )︂ ways, so there are (︂7 1 )︂ · (︂9 2 )︂ cases. If two pairs of chairs are adjacent to each other, we could have(7
-
[41]
By the similar reason above, for each case there are(9
ways to select but we have to subtract six cases where three chairs are adjacent to each other. By the similar reason above, for each case there are(9
-
[42]
If three pairs: 12, 34, 56; 12, 34, 78; 12, 45, 67; 12, 45, 78; 12, 56, 78; 23, 45, 67; 23, 45, 78; 23, 56, 78; 34, 56, 78– in total 10 cases and for each there are (︂9 4 )︂ ways
cases, so in this case we have (︂9 3 )︂ ·15cases. If three pairs: 12, 34, 56; 12, 34, 78; 12, 45, 67; 12, 45, 78; 12, 56, 78; 23, 45, 67; 23, 45, 78; 23, 56, 78; 34, 56, 78– in total 10 cases and for each there are (︂9 4 )︂ ways. If four pairs, only one way and there are(9
-
[43]
crossing
ways as well. Sum everything up the answer is: 2907 89 A Comprehensive Anatomy of Human and DeepSeek-R1 LLM Mathematical Reasoning B.26. AIME_1_26 Let S be the set of vertices of a regular24-gon. Find the number of ways to draw12 segments of equal lengths so that each vertex inSis an endpoint of exactly one of the12segments. R1 Reasoning InSteps 1–5, R1 b...
-
[44]
Losing sight of the problem’s objective, one proceeds with blind calculations.R1 forgets that the problem requires findingt, thus dedicating the majority of the effort to solving forx and producing substantial nonproductive computations
-
[45]
93 A Comprehensive Anatomy of Human and DeepSeek-R1 LLM Mathematical Reasoning Human Reasoning Let|A 1 A2|=aand|A 1 A3|=b
Habitually uses numerical estimation for verification.During numerical verification, √ 5 is often approximated as a decimal to verify the equality of the equations; this approach lacks mathematical rigor. 93 A Comprehensive Anatomy of Human and DeepSeek-R1 LLM Mathematical Reasoning Human Reasoning Let|A 1 A2|=aand|A 1 A3|=b. Forcos (Ai A1 Ai+1) = 12 13, ...
2016
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.