Artificial Intelligence · arXiv:2609.34565 v2 · submitted 28 Sep 2026
FlowState: Execution State as Memory for Long-Horizon LLM Agents
The paper says
“Long-horizon tasks require LLM agents to continually draw on information from earlier interactions.”
more of the abstract
However, retaining the full history increases context costs, while compressing it risks losing details needed later, and the relevance of historical information often becomes apparent as the task progresses. To address these challenges, we propose FlowState, which treats execution state as memory that can be retained and revisited across requests, unifying current decision-making with the reuse of historical information. FlowState preserves semantically typed state nodes, their relations, and references to raw tool observations, separating persistent retention from on-demand access. Within a single execution loop, Incremental State Update (ISU) maintains the current state based on new inputs and feedback, while Progressive State Access (PSA) progressively reveals historical states and supporting evidence as needed during reasoning. Together, these mechanisms enable agents to reassess prior decisions in light of new information and guide subsequent actions. Compared with a full-context baseline using the same DeepSeek-V4-Flash model, FlowState improves the average success rate on MemoryArena and the average pass rate on τ3-Bench by 4.55 and 13.95 percentage points, respectively, while reducing total token consumption by 43.2% and 40.6%. These results demonstrate the performance and efficiency advantages of FlowState on long-horizon tasks.
from the abstract
Pith says
Machine review · deepseek-v4-flash · 29 Sep 2026 · of version 1 of the paper; version 2 is newer · what was read was not recorded · not checked by a person
It can be wrong. Challenge this review or reply to it.
Holds up, with conditions. A solid agent-memory systems paper with honest arithmetic and a genuinely new state-as-revisitable-memory design, but the headline success-rate gain hinges on a single near-zero Web Shopping run and needs per-seed variance before the numbers carry weight. This line is wrong
3 major objections · 4 minor
How often objections from reviews like this one are right: not yet measured
Not checked: figures, code, what the references say.
The paper says, at §3.1–3.2, Table 1
The review did not quote this passage.
Pith says: Objection 1 of 3 · major
The central MemoryArena SR claim rests on count differences that are at or near sampling noise. The 4.55 pp average SR gain decomposes into +5.33 pp on Web Shopping (0/150 vs 8/150), +0.37 pp on Travel Planning (1/270 vs 2/270), +7.50 pp on Math (12/40 vs 15/40), and +5.00 pp on Physics (11/20 vs 12/20).
more of objection 1
Only the Web Shopping difference approaches significance under a Fisher exact test (one-sided p ≈ 0.004), and that p-value does not survive multiple-testing across four environments and seven baselines; the Math, Physics, and Travel deltas are each consistent with noise (p ≈ 0.6, p ≈ 1.0, and p ≈ 1.0, respectively). With temperature 0.1 and no reported seed count, the uncertainty is sampling over the fixed task sets. The paper must report exact trial counts per cell, two-sided binomial confidence intervals, and the number of evaluation runs; if the aggregate SR improvement cannot be shown robust (e.g., via a bootstrap over tasks), the claim should be tempered to the progress-score (PS) improvements, which are large and consistent across all four environments.
The paper says, at §3.2, Table 2
The review did not quote this passage.
Pith says: Objection 2 of 3 · major
The generality claim ("FlowState's benefits extend beyond MemoryArena") is supported only against the full-context baseline: on tau3-Bench, none of the memory-system, state-tracking, or hybrid baselines evaluated on MemoryArena (ReasoningBank, BM25, Mem0, ZipAct, ZipAct+BM25) is run, so the paper cannot show that FlowState outperforms these method families on tau3-Bench.
more of objection 2
The abstract's comparison to full context is consistent with what is reported, but the positioning of FlowState against state-tracking and memory methods in Sections 1 and 5 goes beyond the evidence in Table 2. I recommend either evaluating the strongest one or two baselines on tau3-Bench or explicitly limiting the tau3-Bench claim to the full-context comparison.
The paper says, at §4.1, Table 3
The review did not quote this passage.
A classifier read this sentence of the paper as acknowledging the concern (probability 0.80):
“These ablations assess the overall contribution of each mechanism without isolating the contributions of its individual design choices. 1MemoryArena revision:6cd9de1;τ 3-Bench v1.0.1:fc0055d. 2Mem0 revision:8d6c001. 3ZipAct revision:f025804. 20”
Pith says: Objection 3 of 3 · major
The component ablation is weaker than the surrounding text suggests, even though the paper's sentences are carefully hedged. On Web Shopping the w/o PSA variant has higher PS (44.67 vs 42.89), so the performance benefit of PSA in that environment is confined to SR and token usage; the Travel SR gap it supports (0.74 vs 0.37, i.e., 2 vs 1 of 270 tasks) is noise-scale.
more of objection 3
The w/o ISU variant constructs states only from final responses via fixed rules, and the paper explicitly acknowledges (Appendix B.5) that this does not isolate ISU's design. The claims that "ISU improves task performance" and "PSA supports task success" should be stated with these caveats in the main text rather than only in the appendix, and the ablation should report per-task counts and confidence intervals as in my first comment.
4 minor objections
The paper says, at Figure 3 caption
Pith says: minor objection 1 of 4
The caption reports DeepSeek token reduction of −46.5%, computed over the three tasks in that figure, whereas the abstract and §3.2 report −43.2% over four MemoryArena environments; the difference in task subset should be stated explicitly to avoid reader confusion.
The paper says, at Appendix B.5
Pith says: minor objection 2 of 4
The paper specifies generation settings (temperature 0.1, top_p 1.0) but never states how many independent evaluation runs were performed for any table; this should be stated explicitly, since the near-deterministic setting means the relevant uncertainty is over the fixed task sets.
show 2 more
The paper says, at Reproducibility Statement
Pith says: minor objection 3 of 4
The promise to release source code "at an appropriate time" is vague; committing to a concrete release vehicle or repository would substantially strengthen the reproducibility statement for a paper whose contributions hinge on prompt and runtime details.
The paper says, at Appendix B.3, flowstate_rules
Pith says: minor objection 4 of 4
The prompt instructs the agent to "Never infer exact facts or identifiers from summaries," and the whole mechanism depends on the model emitting well-formed deltas and access requests; reporting the measured frequency of rejected or invalid state-delta/access proposals in the benchmark runs would give readers a concrete sense of how often the runtime guardrails actually fire.
What would settle it
Run the full MemoryArena evaluation (150 shopping / 270 travel / 40 math / 20 physics tasks) for FlowState and DeepSeek-V4-Flash Full Context under 5 independent seeds with the same temperature=0.1 sampling, and report SR/PS per task with binomial 95% CI and a paired permutation test on task-level success. If the 5.33pp Web Shopping SR gain is not significant at p<0.05 under seed variation, the headline MemoryArena SR claim weakens to 'comparable or mixed SR with large gains on Progress Score and token savings only'. If instead SR gains are stable across seeds, the concern is resolved. Also report the exact count of Shopping tasks that succeeded under FlowState.
from the skeptic's reading
How the review went
4 readings by deepseek-v4-flash · 29 Sep 2026
- Reader holds up, with conditions
- Skeptic partly agrees
- Editor wrote the verdict above
- Referee major revision: 3 major objections, 4 minor
Not run on this paper: the simulated authors' defence.
Pith did not check this review's objections against the paper.
The review in full
The review in brief, machine-written
FlowState is a solid systems paper. It treats execution state as persistent, individually addressable, revisitable memory with progressive disclosure, and the headline numbers are honestly computed: the 4.55 pp SR gain, the 13.95 pp pass-rate gain, and the 43.2% and 40.6% token savings all check out against Tables 1 and 2. The ablations are meaningful, the limitations (safety, excluded Progressive Web Search, the Retail Action Accuracy decline, ablation scope) are stated rather than hidden, and the writing is direct.
What is new: the unification of state tracking and retrieval memory in one representation, with Incremental State Update and Progressive State Access playing complementary roles. The persistent repository / historical index / active state split, with relation-linked evidence references, is a real step beyond ZipAct and the BM25/Mem0 baselines. The per-subtask-depth analysis is a nice addition, and the ablation design sensibly isolates the two mechanisms.
Soft spots, in order of seriousness. The MemoryArena SR claim is statistically fragile. The 5.33 pp gain on Bundled Web Shopping is 8/150 versus 0/150; Group Travel Planning 0.74 versus 0.37 is one or two tasks on 270; Physics 60 versus 55 is one task on 20. There are no seeds, no variance, and no significance test anywhere. The stress-test note is right: the entire SR superiority over the memory baselines collapses without that one shopping score. The tau3-Bench pass-rate gains are sturdier by raw counts, so the cross-benchmark claim is less fragile, but the paper needs per-seed reporting.
Second, the mechanism depends on the model emitting correct state deltas and access requests. The runtime restricts disclosure to IDs(Gk) union ToolRefs(Sk,t), and the prompt forbids inferring facts from summaries, but nothing tests how often the model obeys or what happens when it does not. The authors acknowledge that outdated or incorrect assumptions can persist into later decisions, and historical nodes cannot be deleted. That is a real deployment concern, not a hidden one.
Third, MAGE is described but not compared experimentally, the backbone models are future-dated and unverifiable, and the code is promised rather than released. None of these is fatal, but they cap how much weight the current numbers can carry.
This paper is for people building production LLM agents who want a concrete, efficient memory-and-state design with benchmark numbers. It deserves a serious referee. My recommendation: send it to peer review, and require per-seed variance, code release or a detailed reproducibility appendix, and ideally a direct MAGE comparison before acceptance.
The strongest objection
The claim's load-bearing evidence is Table 1's SR column. On Bundled Web Shopping, FlowState SR is 5.33% versus 0.00% for all DeepSeek baselines, a gain of only 3 tasks if the set is 150 tasks, and there is no reported variance, seed count, or significance test anywhere in Sections 3-4 or Appendix B.5. The reader flagged the near-zero SR environments but did not make it primary: the paper's arithmetic check is about consistency, not statistical stability, and a 5.33pp gain on n=150 with observed 0/150 versus 8/150 is borderline (p≈0.004 one-sided exact, so it passes a naive Fisher test, but the meaningful test is whether the deterministic runs would be stable under seed/code perturbation). Travel Planning SR 0.74 vs 0.37 is 1-2 tasks on 270. Physics SR 60.00 vs 55.00 is 1 task on 20. The 13.95pp tau3-Bench pass-rate gain is more robust (39 vs 29 of 50 Airline; 93 vs 84 of 114 Retail), so the cross-benchmark claim is less fragile, but the headline 4.55pp MemoryArena SR gain is dominated by a 3-task Poisson-noise-scale difference. A single-seed trace that adds one travel task or two math tasks would not change the paper's story, whereas the entire SR-based superiority over memory baselines collapses without that one score.
Most of the reader's CONDITIONAL verdict is justified: arithmetic is internally consistent, ablations isolate ISU/PSA, limitations are stated, but the code is unreleased, error bars are missing, and the benchmark subset is disclosed. My stress-test identifies which unresolved item is most load-bearing: not the code, and not benchmark coverage in general, but the specific fact that the primary MemoryArena SR claim (Table 1) is carried by one near-zero-signal result — Web Shopping 0.00 to 5.33 SR — whose absolute effect is a handful of tasks and whose robustness to seed/repetition is never established. This differs from the reader's framing: the reader lists missing variance as one of several mechanical gaps, while I argue it is the single point on which the central cross-benchmark claim could fail even if the paper's implementation is exactly as described. The tau3-Bench pass-rate gain (13.95pp) is larger relative to its task counts and would survive a plausible seed shift, so my concern does not push toward REJECT: the claim that FlowState helps may be true, but the MemoryArena SR evidence is near a coin-flip threshold. I therefore keep the verdict CONDITIONAL (same as reader) but with a sharper, testable condition: per-seed SR stability on the 4 MemoryArena task sets. Remaining issues (no code release, excluded PWS, no MAGE comparison) are real but secondary to the statistical fragility estimate.
The referee's report
The core idea — using the execution state itself as the unit of long-term memory, with incremental updates and progressive, evidence-linked access — is a sensible and well-specified proposal; if the empirical claims hold, it would be a useful result for the agent-memory literature. Strengths of the manuscript include the careful formalization of the runtime (serialization, validation rules Eq. (1)–(5), Algorithm 1), the disclosure of benchmark and implementation details (Appendix B.3–B.5), reproducible-looking baseline choices, and honest statements of scope, including the acknowledged limitation that the ablations do not isolate individual design choices and that benchmark evaluation does not establish deployment safety. The main weakness is that the headline MemoryArena success-rate claim is statistically fragile: several per-environment SR deltas are differences of one to three tasks on small or near-zero-success sets, and the paper reports no confidence intervals, significance tests, or repeat runs anywhere in Sections 3–4.
Pith's reading of the paper (a machine's summary, not the authors')
The paper tries to establish that an LLM agent can keep its working context compact over long, multi-session tasks without losing the details it will need later, by treating execution state itself as a persistent memory unit that survives across requests and can be revisited on demand. Full interaction histories are expensive and bury relevant details; compressed states risk discarding what is needed; external memory only helps if the right record is retrieved at the right moment. FlowState's answer is to let the agent build a semantically typed execution state, retain it in a repository outside the context, and expose historical states progressively through an index of identifiers and summaries as the need emerges during reasoning. If the central claim holds, one mechanism does double duty: it maintains the current decision state and it reuses prior evidence, without re-reading the whole history. On the two benchmarks evaluated, the paper reports that this yields both higher success and lower token consumption than a full-context baseline operating on the same model.
What carries the argument. The central object is a persistent state repository of semantically typed nodes covering preferences, knowledge, attributes, judgements, and artifacts, each carrying an identifier, a summary, a structured value, and directed relations to other nodes and to raw tool observations. Two operations drive it inside a single ReAct-style loop: Incremental State Update (ISU), which commits Add, Update, and Remove deltas to the current-request state without regenerating everything, and Progressive State Access (PSA), which uses a lightweight Historical State Index of identifiers and summaries to disclose historical nodes and their linked evidence on demand. A runtime validation layer restricts updates to nodes created during the current request and restricts access to indexed identifiers and referenced tool results, so the model's own choices about what to write into the state and which historical nodes to visit are the only contact between the mechanism and the content it retains.
Core claim. The central claim is that persistent, individually addressable execution state can serve simultaneously as the working state for the current request and as cross-request memory, and that separating persistent retention from context visibility resolves the trade-off between full-history context cost and compression loss. Using the same DeepSeek-V4-Flash model, FlowState reports improving the average success rate on MemoryArena by 4.55 percentage points and the average pass rate on tau3-Bench by 13.95 percentage points while reducing total token consumption by 43.2% and 40.6% relative to a full-context baseline. The two mechanisms carry complementary load: ablations show that removing Incremental State Update drops success rate to zero in both tested environments, while removing Progressive State Access raises token usage and lowers performance in travel planning, supporting the claim that on-demand disclosure, rather than exposure of all history, drives the gains. The advantage also holds across three base models and persists at later subtask depths, where most baselines approach zero accuracy.
The premise it rests on. Everything rests on the model reliably producing correct state nodes and correct access requests: a wrong summary written into the index, or a wrong choice of which historical node to disclose, places incorrect or hallucinated facts into later decisions, and since historical states cannot be deleted, erroneous nodes persist to mislead subsequent steps, a risk the authors themselves acknowledge in their ethics statement.
If the paper is right (Pith's inferences, not the authors')
If the claim holds, agents can maintain a compact active context while keeping historical detail reachable, which directly lowers the token cost of long-horizon work.
Reconsidering an earlier decision in light of new information, such as a price drop that changes whether a cancellation fee is worth paying, happens by restoring the linked evidence rather than rescanning the full history.
On tasks with interdependent subtasks, the gains concentrate at later subtask depths where other baselines approach zero accuracy, suggesting the method's value grows with task length.
Because the same representation covers current decisions and historical reuse, the framework extends beyond shopping, travel, and formal reasoning to policy-constrained service interactions, as demonstrated by the tau3-Bench results.
A testable prediction, left implicit in the paper, is that the performance and token-efficiency gap over full context widens as the number of subtasks per task grows, because the savings come precisely from not re-reading accumulated history.
The framework shifts the hard part of agent prompting from organizing history to teaching the model when an access request is warranted; isolating that decision through prompt-variation studies would show how much of the gain is access discipline versus state quality.
Because historical states cannot be deleted and can only be superseded by new nodes, the method inherits the staleness problem identified in prior work: outdated states persist and can mislead later steps. Detecting or expiring stale nodes rather than merely annotating them would be a natural extension the paper leaves open.
The reported token savings suggest the approach could translate into measurable cost reduction in deployed multi-session agents, where per-request context length currently dominates inference expense.
Check this review
Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.
How often reviews like this one are right
Reviews like this one: written by deepseek-v4-flash, where what was read was not recorded.
How often objections from reviews like this one are right: not yet measured
Reviews by people
This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.
The paper
Abstract
Long-horizon tasks require LLM agents to continually draw on information from earlier interactions. However, retaining the full history increases context costs, while compressing it risks losing details needed later, and the relevance of historical information often becomes apparent as the task progresses. To address these challenges, we propose FlowState, which treats execution state as memory that can be retained and revisited across requests, unifying current decision-making with the reuse of historical information. FlowState preserves semantically typed state nodes, their relations, and references to raw tool observations, separating persistent retention from on-demand access. Within a single execution loop, Incremental State Update (ISU) maintains the current state based on new inputs and feedback, while Progressive State Access (PSA) progressively reveals historical states and supporting evidence as needed during reasoning. Together, these mechanisms enable agents to reassess prior decisions in light of new information and guide subsequent actions. Compared with a full-context baseline using the same DeepSeek-V4-Flash model, FlowState improves the average success rate on MemoryArena and the average pass rate on τ3-Bench by 4.55 and 13.95 percentage points, respectively, while reducing total token consumption by 43.2% and 40.6%. These results demonstrate the performance and efficiency advantages of FlowState on long-horizon tasks.
The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.
abstract compiled · 2026-09-29
The authors' main claim, as the classifier picked it out 0.98
“These results demonstrate the performance and efficiency advantages of FlowState on long-horizon tasks.”
How the main claim is stated: asserted 1.00
What the abstract itself states, as probabilities: claims first 0.04 · states limitation 0.11 · human participants 0.04 · controlled comparison 0.97 · randomized assignment 0.01 · states quantitative result 0.99 · compares with prior results 0.96 · mentions code or data availability 0.03
The abstract, sentence by sentence
This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.
| Sentence | Role | The authors' own work | States the main conclusion | Strength |
|---|---|---|---|---|
| Long-horizon tasks require LLM agents to continually draw on information from earlier interactions. | background 1.00 | 0.05 | 0.02 | asserted 1.00 |
| However, retaining the full history increases context costs, while compressing it risks losing details needed later, and the relevance of historical information often becomes apparent as the task progresses. | background 0.75 | 0.07 | 0.02 | asserted 0.91 |
| To address these challenges, we propose FlowState, which treats execution state as memory that can be retained and revisited across requests, unifying current decision-making with the reuse of historical information. | contribution 0.98 | 0.70 | 0.13 | asserted 0.98 |
| FlowState preserves semantically typed state nodes, their relations, and references to raw tool observations, separating persistent retention from on-demand access. | method 0.95 | 0.80 | 0.04 | asserted 1.00 |
| Within a single execution loop, Incremental State Update (ISU) maintains the current state based on new inputs and feedback, while Progressive State Access (PSA) progressively reveals historical states and supporting evidence as needed during reasoning. | method 0.97 | 0.80 | 0.05 | asserted 1.00 |
| Together, these mechanisms enable agents to reassess prior decisions in light of new information and guide subsequent actions. | interpretation 0.43 | 0.56 | 0.14 | asserted 0.97 |
| Compared with a full-context baseline using the same DeepSeek-V4-Flash model, FlowState improves the average success rate on MemoryArena and the average pass rate on $\tau^3$-Bench by 4.55 and 13.95 percentage points, respectively, while reducing total token consumption by 43.2% and 40.6%. | result 1.00 | 0.95 | 0.29 | asserted 1.00 |
| These results demonstrate the performance and efficiency advantages of FlowState on long-horizon tasks. | interpretation 1.00 | 0.83 | 0.79 | asserted 1.00 |
What kind of paper the classifier reports this is: method or tool 1.00 · evidence from observation or experiment 0.94
What was examined, and what was not
Sources read
- arXiv listing abstract v1
Not examined
- full text
- figures and tables
- references and citing sentences
- relations to other papers
Levels run
- P abstract
Not run
- S sections
- T body sentences
- C citations
- R cross-paper relations
How the labels were made
Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-09-29: 1 request, 6,396 input tokens.
Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON
Figures (4)
Figures are the authors', reproduced from the paper.
Cited by 2 Pith papers
Reviewed papers on Pith that cite this one. The full citation record
- CarryOn: Learning What to Keep in Context for Long-Horizon LLM Agents cs.CL · Nov 2026
- Who Schedules Agent Memory? A Sidecar Approach to Long-Horizon Tasks cs.AI · Oct 2026
Challenges and discussion
Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.
Come back by email
A short list of the week's newly reviewed papers. Nothing is sent until you confirm from your inbox, and every email has a link that stops it. No account.
No comments yet.
Sign in to comment