Pith.

Computation and Language · arXiv:2609.02702 v2 · submitted 2 Sep 2026

Trace as State: Reasoning Traces as Conditional States for Long-Context Transformers

Xu Zou, Jie Tang

The paper says

“Transformers process information causally, but long-context reasoning may depend on task state discovered only later. We formalize this mismatch through conditional state update tasks.”

more of the abstract

For causal state update processors, providing the condition first can require exponentially less memory in the worst case than providing it last. Motivated by this principle, we introduce Trace as State. We use collected reasoning traces as a textual proxy for task state and place it before the long-context block on a fresh pass, allowing information derived previously to guide rereading. We conduct extensive experiments on Trace as State and Trace Append, a matched control that uses the same task state proxy but put it after the context. Across three models and three long-context datasets, Trace as State outperforms Trace Append in 26 of 27 reported combinations of model, task, and metric. On GraphWalks Parents, exact match lifts DeepSeek V4 Pro Preview from 29.2% on the initial pass and 43.0% with Trace Appendto 81.8% with Trace as State, and from 66.4% and 83.2% to 100.0% for GLM-5.2. These results show that placing traces before the context can improve long-context reasoning while retaining the causal transformer structure.

from the abstract

Pith says

Machine review · deepseek-v4-flash · 8 Sep 2026 · of version 1 of the paper; version 2 is newer · what was read was not recorded · not checked by a person

It can be wrong. Challenge this review or reply to it.

Holds up. Clean, well-controlled empiricism: placing reasoning traces before the long context consistently outperforms placing them after. The paper is honest, thorough, and the main result holds up. This line is wrong

2 major objections · 5 minor

How often objections from reviews like this one are right: not yet measured

Are you an author? What you can do.

Not checked: figures, code, what the references say.

The paper says, at §3.1, Eq. (2)

The review did not quote this passage.

Pith says: Objection 1 of 2 · major

The worst-case memory bound for condition-last processing is given as 'l log2|S||S| m' which appears to be a typesetting error; it should read ⌈log₂(|S|^|S|)⌉. While this does not affect the conceptual argument, the presentation should be corrected.

The paper says, at §3.2

The review did not quote this passage.

Pith says: Objection 2 of 2 · major

The paper states that 'causal transformers with finite context length and finite precision are causal state update processors as their max memory are bounded.' The statement is technically correct for the abstraction, but the theoretical analysis in §3.1 assumes a deterministic, exact processor, while LLMs are stochastic and approximate.

more of objection 2

The authors acknowledge that the conditional state update tasks are not literal models of real world tasks. However, to avoid potential over-interpretation, the theoretical section could more explicitly delimit the scope of the analogy (e.g., noting that the exponential separation applies to worst-case tasks under an exact finite-state abstraction, not to the specific benchmarks evaluated). This is a presentational suggestion, not a flaw in the empirical results.

This review says the paper's mathematics is wrong. Pith did not check that against the paper's source.

5 minor objections

The paper says, at Table 1

Pith says: minor objection 1 of 5

The symbol 'n_tr' is used in the table, but in §3.2 the subscript is written as 'n tr' with a space. Please unify the notation (e.g., n_tr throughout).

The paper says, at §4.1

Pith says: minor objection 2 of 5

The description of the 'xhigh' prompt for Qwen 3.7 Max would be clearer if the exact prompt text were included in Appendix C.2, which already contains model-specific details.

show 3 more

The paper says, at Appendix F, Table 6

Pith says: minor objection 3 of 5

The unit 'millions' for token counts should be explicitly stated in the table header or caption for clarity.

The paper says, at §4.3, Table 3

Pith says: minor objection 4 of 5

The 'Answer Feedback' condition uses first-pass answers a = [a_1,...,a_{n_tr}], but it is not completely clear how multiple answers are serialized (e.g., with delimiters). Please specify the serialization format, even briefly, in the main text or appendix.

The paper says, at §5 (Limitations)

Pith says: minor objection 5 of 5

The limitation about reduced key–value cache reuse is noted, but the extent of the overhead (e.g., typical factor of 2 in cost) could be quantified from the token usage in Appendix F for practical guidance.

What would settle it

Independently reproduce the full Table 2 results for one model-task pair (e.g., DeepSeek V4 Pro on GraphWalks 256K Parents) using the same API, prompt templates, and scoring code but with a fresh run and new random seeds. If the Trace as State minus Trace Append gap remains directionally consistent (positive on EM and F1, magnitude within 10 percentage points of the reported values), the core finding is robust to run-to-run variability.

from the skeptic's reading

How the review went

4 readings by deepseek-v4-flash · 8 Sep 2026

  1. Reader holds up confidence high
  2. Skeptic agrees 42 s · 20,159 tokens read, 4,779 written
  3. Editor wrote the verdict above 14 s · 20,988 tokens read, 1,329 written
  4. Referee minor revision: 2 major objections, 5 minor 28 s · 20,448 tokens read, 2,822 written

Not run on this paper: the simulated authors' defence.

Pith did not check this review's objections against the paper.

The review in full

The review in brief, machine-written

This is a solid, honest paper. The core idea—collect reasoning traces from a first pass, then put them before the long context on a second pass—is not revolutionary in isolation, but the systematic comparison of trace-before versus trace-after is new and well executed. The authors isolate the placement effect by keeping the trace text T identical across conditions, and they run the comparison across three models, three benchmarks, and 27 metric–model–task combinations. The result is clean: 26 of 27 favor Trace as State over Trace Append. They also run a sensible set of ablations (random traces, answer feedback, Re2, trace only) that rule out the obvious confounds. The one negative case (GLM-5.2 BFS F1) is honestly reported and small. The theoretical motivation drawn from conditional state update tasks adds a useful lens, though it is explicitly informal and not load-bearing—which is fine; the paper stands on its experiments.

Soft spots are real but not fatal. Trace truncation at 50,000 characters may limit performance on the longest traces, but this is a practical constraint, not a design flaw. The NUB-1M metric uses model-judged accuracy, which is weaker than human evaluation, though the authors report manual verification with no errors found. The method adds substantial latency and cost (a second full pass plus the initial trace generation), so it is not free. But the paper acknowledges this in its limitations section.

This is not a paradigm-changer, but it is a careful, reproducible demonstration that trace placement matters and that feeding traces back before the context is generally beneficial. For anyone working on inference-time scaling, long-context retrieval, or prompt engineering, this is a useful data point. The paper deserves a proper peer review. I would send it out without hesitation.

The strongest objection

I have carefully reviewed the paper, the reader's verdict, and the full text including appendices. The central claim—that placing reasoning traces before the long context (Trace as State) improves long-context reasoning over placing them after (Trace Append)—is empirically well-supported by 26 of 27 comparisons across three models, three tasks, and multiple metrics. The paper's weakest assumption (that traces serve as a useful textual proxy for task state) is explicitly acknowledged and tested via multiple strong ablations (Random Trace, Answer Feedback, Question First, Re2, Trace Only), all of which are consistent with the interpretation that the benefit arises from the trace carrying task-relevant information that guides context re-processing. The one negative case (GLM-5.2 GraphWalks BFS F1) is minor and within expected variance. The theoretical analysis is explicitly presented as motivation, not as a literal model of transformer behavior, so it is not load-bearing. Potential confounds (e.g., question-to-context distance differing between conditions) are addressed by the Re2 control, which also places content between context and question and still underperforms Trace as State. No internal inconsistency, undisclosed limitation, or methodological flaw undermines the central result. The paper is thorough and honest. I do not find a load-bearing concern that would alter the verdict.

The reader identified the weakest assumption as whether reasoning traces serve as a useful textual proxy for task state. I agree this is the most natural concern. However, the paper provides extensive evidence that the assumption holds in practice: (i) 26/27 comparisons favor Trace as State; (ii) Random Traces hurt performance, so the effect is not a generic formatting artifact; (iii) Answer Feedback and Question First improve less than Trace as State, showing that traces carry more state-relevant information than answers or questions alone; (iv) the trace-count ablation shows monotonic improvement with more traces, consistent with richer state information. These controls convincingly address the concern. I considered other potential issues—the distance of the question from the context, the faithfulness of traces, the applicability of the theoretical bound—but each is either explicitly addressed, minor, or not load-bearing. The paper's honesty about limitations and its thorough experimental methodology mean the verdict of ACCEPT is well-justified with no adjustment needed.

The referee's report

The paper demonstrates a simple, general, and empirically robust method for improving long-context reasoning in causal transformers without architectural changes. If the results hold across broader settings, the method offers a practical inference-scaling technique that leverages the growing availability of reasoning traces from modern LLMs. The experimental design is rigorous: 5 repeats per condition, confidence intervals, same trace T for both placement conditions, and multiple ablations isolating the placement effect. The honest reporting of one negative case (GLM-5.2 BFS F1) and the clear acknowledgement of limitations (need for access to traces, increased cost, truncation at 50,000 characters) strengthen the paper's credibility.

Pith's reading of the paper (a machine's summary, not the authors')

The paper argues that causal transformers face a structural mismatch when task-relevant state (like an active target or search frontier) is discovered only after processing long contexts. It formalizes this mismatch through a conditional state update abstraction, showing that placing the condition (task state) before the information sequence can require exponentially less memory in the worst case than placing it after. Based on this principle, the authors propose Trace as State, a method that collects reasoning traces from a first pass and feeds them as a textual state proxy before the long context in a second causal pass. Across three frontier models and three long-context benchmarks, Trace as State outperforms a matched control (Trace Append) in 26 of 27 settings, with gains of up to 53 points on GraphWalks Parents. The results suggest that the ordering of task state relative to context matters greatly for causal long-context reasoning, and that reasoning traces can serve as an effective proxy for that state.

What carries the argument. The core machinery is the conditional state update abstraction: a causal processor updates a state $s_i = U(s_{i-1}, c_i)$ as it reads each unit $c_i$. When the initial state (condition) is known first, the processor tracks one state path; when it arrives last, the processor may need to retain a function from every possible condition to the resulting state, leading to an exponential worst-case memory gap. Reasoning traces serve as an imperfect textual proxy for this condition, and Trace as State places the serialized traces before the long context in a fresh causal pass, making the state available during rereading.

Core claim. The central discovery is that for causal transformers, extracting task-relevant state from reasoning traces and placing it before the long context in a second pass substantially improves reasoning performance compared to placing it after. This is grounded in a theoretical analysis of conditional state update tasks, where a condition-first order can require only $\log|S|$ bits of memory, while a condition-last order may require $|S|\log|S|$ bits in the worst case. The empirical evidence shows this ordering advantage holds across models and tasks: Trace as State beats Trace Append in 26/27 comparisons, with the largest gains on tasks requiring state tracking like GraphWalks Parents.

The premise it rests on. The reasoning traces generated by the model capture task-relevant state information useful for guiding a second pass; if they are incomplete, lossy, or incorrect (as the paper acknowledges), the method's advantage diminishes.

If the paper is right (Pith's inferences, not the authors')

Long-context reasoning can be improved on existing causal transformers without architectural changes, simply by reusing reasoning traces in the right order.

The ordering principle suggests a general design pattern for inference scaling: extract task-state proxies from earlier passes and place them before the input in later passes.

The benefit is likely to grow with task complexity where state tracking is essential, as seen in GraphWalks Parents versus BFS.

Training procedures could be designed to optimize the content and granularity of reasoning traces for state feedback, potentially further improving the gain.

The conditional state update abstraction may extend to non-textual state proxies (e.g., latent representations or recurrent states), suggesting that similar gains could be achieved with model internals rather than text.

The method implicitly creates a form of iterative refinement without architectural modification; this could be combined with other inference-time techniques like self-consistency or tree-of-thoughts.

The strong results on GraphWalks (a synthetic graph task) indicate the method is particularly effective for tasks requiring maintaining distributed state across long inputs; real-world applications like document question answering or code understanding may also benefit.

Check this review

Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.

How often reviews like this one are right

Reviews like this one: written by deepseek-v4-flash, where what was read was not recorded.

How often objections from reviews like this one are right: not yet measured

Every value above, in the review's public record (JSON)

Reviews by people

This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.

Request a human review

How Pith checks papers · The full audit of this paper

The paper

Abstract

Transformers process information causally, but long-context reasoning may depend on task state discovered only later. We formalize this mismatch through conditional state update tasks. For causal state update processors, providing the condition first can require exponentially less memory in the worst case than providing it last. Motivated by this principle, we introduce Trace as State. We use collected reasoning traces as a textual proxy for task state and place it before the long-context block on a fresh pass, allowing information derived previously to guide rereading. We conduct extensive experiments on Trace as State and Trace Append, a matched control that uses the same task state proxy but put it after the context. Across three models and three long-context datasets, Trace as State outperforms Trace Append in 26 of 27 reported combinations of model, task, and metric. On GraphWalks Parents, exact match lifts DeepSeek V4 Pro Preview from 29.2% on the initial pass and 43.0% with Trace Appendto 81.8% with Trace as State, and from 66.4% and 83.2% to 100.0% for GLM-5.2. These results show that placing traces before the context can improve long-context reasoning while retaining the causal transformer structure.

The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.

abstract compiled · 2026-09-23

The authors' main claim, as the classifier picked it out 1.00

“These results show that placing traces before the context can improve long-context reasoning while retaining the causal transformer structure.”

How the main claim is stated: asserted 0.98

What the abstract itself states, as probabilities: claims first 0.06 · states limitation 0.14 · human participants 0.03 · controlled comparison 0.98 · randomized assignment 0.02 · states quantitative result 0.99 · compares with prior results 0.96 · mentions code or data availability 0.03

The abstract, sentence by sentence

This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.

SentenceRoleThe authors' own workStates the main conclusionStrength
Transformers process information causally, but long-context reasoning may depend on task state discovered only later. background 0.75 0.08 0.03 hedged possibility 0.93
We formalize this mismatch through conditional state update tasks. contribution 0.71 0.90 0.03 asserted 0.97
For causal state update processors, providing the condition first can require exponentially less memory in the worst case than providing it last. background 0.49 0.47 0.10 asserted 0.91
Motivated by this principle, we introduce Trace as State. contribution 0.96 0.76 0.05 asserted 0.77
We use collected reasoning traces as a textual proxy for task state and place it before the long-context block on a fresh pass, allowing information derived previously to guide rereading. method 0.90 0.92 0.08 asserted 0.97
We conduct extensive experiments on Trace as State and Trace Append, a matched control that uses the same task state proxy but put it after the context. method 0.87 0.96 0.04 asserted 0.71
Across three models and three long-context datasets, Trace as State outperforms Trace Append in 26 of 27 reported combinations of model, task, and metric. result 1.00 0.96 0.28 asserted 1.00
On GraphWalks Parents, exact match lifts DeepSeek V4 Pro Preview from 29.2% on the initial pass and 43.0% with Trace Appendto 81.8% with Trace as State, and from 66.4% and 83.2% to 100.0% for GLM-5.2. result 1.00 0.95 0.11 asserted 1.00
These results show that placing traces before the context can improve long-context reasoning while retaining the causal transformer structure. interpretation 1.00 0.92 0.89 asserted 0.98

What kind of paper the classifier reports this is: method or tool 0.99 · evidence from observation or experiment 1.00

What was examined, and what was not

Sources read

  • arXiv listing abstract v1

Not examined

  • full text
  • figures and tables
  • references and citing sentences
  • relations to other papers

Levels run

  • P abstract

Not run

  • S sections
  • T body sentences
  • C citations
  • R cross-paper relations

How the labels were made

Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-09-23: 1 request, 6,917 input tokens.

Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON

Figures (6)
Figure 1 of the paper
Figure 1. Overview for TRACE AS STATE (A) A causal processor carries one running state. (B) In a worst￾case conditional state update task, condition-first processing tracks only the realized state, whereas condition-last processing can require exponentially more working memory. (C) Reasoning trace T provides a textual proxy for task state. (D) TRACE AS STATE places T before the context in a fresh causal pass. ReContext recursively builds and replays an evi￾dence pool for the current query (Aghajohari et al., 2026; Zhao et al., 2026). These methods show that feeding task-relevant text back to…
Figure 2 of the paper
Figure 2. Trace-count ablation on DeepSeek V4 Pro GraphWalks 256K. Panel A reports BFS F1 and Panel B reports Parents F1. Shading shows 95% percentile confidence intervals. AS STATE–TRACE APPEND ordering persists from ntr = 1 through ntr = 5. 5 Conclusion We introduced TRACE AS STATE, an inference ap￾proach that reuses task state information carried in reasoning traces as a textual proxy on a fresh pass over a long context task. Its motivation comes from theoretical analysis of conditional state up￾date tasks: for causal state update processors, a condition available before an information se
Figure 3 of the paper
Figure 3. GraphWalks exact-match gains over the first pass by BFS depth
Figure 4 of the paper
Figure 4. Set F1 gains for the same BFS (top) and Parents (bottom) stratifications. Set F1 records partial overlap
Figure 5 of the paper
Figure 5. Uncertainty for the strict same-T comparisons. The top two rows show GraphWalks 256K exact match and set F1, split into BFS and Parents; the bottom two wide rows show MRCRv2 8-needle exact match and SequenceMatcher ratio, split into 256K and 512K bins. The final panel shows NUB-1M accuracy. Points are means; bars are 95% percentile intervals after averaging 5 repeats within each problem. 19
Figure 6 of the paper
Figure 6. Uncertainty for DeepSeek V4 Pro GraphWalks 256K control comparisons in exact match and set F1.

Figures are the authors', reproduced from the paper.

Cited by 2 Pith papers

Challenges and discussion

Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.

No comments yet.

Sign in to comment