{"id":"ad5e8e10-1aab-4afb-b88e-1a12daa2a53f","arxiv_id":"2608.09044","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"ToE organizes agent experience as a hierarchical tree of reasoning perspectives with outcome-calibrated reliability, and reports gains over experience-free baselines on Game of 24 and FinEvolveBench.","lead":"The paper introduces Tree-of-Experience (ToE), a method that stores an AI agent's past reasoning insights in a structured tree, so the agent can reuse what worked and avoid what failed. On a math puzzle and a financial news benchmark, ToE reports higher accuracy and lower compute, but the financial benchmark is self-created and its raw data is not public.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"On Game of 24, ToE's experience is a verifier-confirmed suffix cache keyed by canonical number states; the ToT baseline has no such cache, so the 20.4-point gain may be caching, not hierarchical experience management.","rationale":"The reader's verdict was CONDITIONAL, and the weakest assumption identified was that the baselines are not equally strong or fairly tuned, specifically the verified-suffix cache in Game of 24 and the benchmark-structure alignment in FinEvolveBench. I agree with that identification for Game of 24 and believe it is the most load-bearing issue for the paper's central quantitative claim. The appendix makes the caching mechanism explicit: states are canonicalized multisets, successful suffixes are stored verbatim after verifier confirmation, retrieval uses exact state matching plus overlap, and the binary update rule means no graded reliability is learned. This is exactly memoization. Without a baseline that is allowed the same memo, the accuracy and efficiency comparison measures memory versus no memory rather than the proposed hierarchical experience-management framework. I do not see an internal inconsistency in the method, and the paper is transparent about the specialization; the problem is one of external validity. The FinEvolveBench results are also hard to audit because the raw data are not released and the benchmark is authored by the same group, but the Game of 24 confound is more crisply testable and is sufficient to cast doubt on the strongest claim. If the proposed ablation shows cached ToT matches ToE, the paper's contribution would need to be reframed: the Game of 24 result would demonstrate a caching benefit, and the central evidence would rest entirely on FinEvolveBench. If cached ToT does not match, then the concern is resolved and the Game of 24 result would support the framework. Since the reader already conditioned acceptance on addressing exactly this issue, I do not move the verdict; it should remain CONDITIONAL pending the check.","tokens_in":16907,"tokens_out":4038,"duration_ms":40308,"concrete_test":"Implement a ToT+Cache baseline on Game of 24 that uses the same canonical multiset state key, the same deterministic arithmetic verifier, and the same repository admission rule (store only verifier-confirmed suffixes) as ToE, but with no hierarchical tree, no reranker, and no reliability scoring; give it the same initial solved puzzles or allow it to cache online, and compare accuracy and LLM calls on the same test set. If ToT+Cache matches ToE's 85.3% accuracy and roughly 11 calls, the reported advantage is attributable to caching rather than to ToE's hierarchical experience mechanism.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's appendix (Details in Game of 24) specifies that an analytical perspective is instantiated as a canonicalized subproblem state, that successful solution suffixes are stored at nodes after verifier confirmation, and that retrieval first considers an exactly matched canonical number state and may additionally use number-set overlap. With eta_G24 = 1, the reliability update collapses to Q_new(e) = r_t in {0,1}; no graded calibration remains. Thus the Game of 24 specialization of ToE is a transposition table / memoization scheme: it caches verified continuations from previously solved puzzles and reuses them when the same or overlapping multiset recurs. The experience-free ToT baseline searches from scratch on every puzzle, making 51.7 calls per puzzle. The reported 85.3% vs 64.9% accuracy and 11.0 vs 51.7 calls could therefore be explained entirely by the presence of a verified-solution cache, not by hierarchical analytical perspectives or reliability-driven experience management. Since this is one of the two benchmark pillars of the abstract's central claim, the headline improvement is not yet established as a property of ToE qua experience-management framework.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Tree-of-Experience (ToE), a framework that organizes an LLM agent's accumulated experience as a hierarchical tree of analytical perspectives and reasoning paths, with reliability scores updated from environmental feedback. The method is evaluated on Game of 24 (with Tree of Thoughts as the reasoning backbone) and on FinEvolveBench, a financial sentiment-analysis benchmark. The authors report that ToE improves Game of 24 accuracy from 64.9% (ToT) to 85.3% while reducing LLM calls from 51.7 to 11.0 per puzzle, and improves tsIC by an average of 41.24% over the experience-free pipeline across 12 settings on FinEvolveBench. They also report that conventional memory methods (Mem0, MemRL, ReMe) often underperform experience-free baselines.","tokens_in":17215,"tokens_out":7819,"duration_ms":64317,"significance":"If the empirical results hold, ToE would be a valuable contribution to experience abstraction for LLM agents, combining structured representation, feedback attribution, and retrieval efficiency. The paper includes explicit prompt templates, implementation details, and a released code package, which are strengths for reproducibility. However, the current evidence is not sufficient to support the central claim because of a likely caching confound in the Game of 24 benchmark and missing uncertainty and reproducibility information in the FinEvolveBench evaluation. The core idea of aligning experience organization with the reasoning structure is interesting and worth further work, but the paper needs substantial additional experiments to substantiate its headline improvements.","major_comments":[{"comment":"The Game of 24 implementation described in the Appendix is effectively a transposition table rather than a hierarchical experience-management system. The appendix states that a successful solution suffix is stored at a node and reused when the same or a sufficiently overlapping state is encountered again, and that retrieval first considers an exactly matched canonical number state. With eta_G24 = 1 (Eq. A4), the reliability update collapses to Q_new(e) = r_t in {0,1}, and only verifier-confirmed continuations are admitted. This means the ToE agent carries a verified-suffix cache across puzzles, while the experience-free ToT baseline (Section 'Performance on Game of 24', Table 2) searches from scratch on every puzzle. The reported 20.4-point accuracy gain and 78.7% call reduction could therefore be entirely explained by caching, rather than by the proposed hierarchical perspective alignment or reliability calibration. The authors should add a control baseline that gives ToT the same verified-suffix cache (e.g., a standard transposition table) and show that ToE still provides a meaningful gain beyond the cache.","section":"Appendix, Details in Game of 24"},{"comment":"The FinEvolveBench results in Table 1 lack variance information: the paper states that each method is run three times and averaged, but no standard deviations, confidence intervals, or significance tests are reported. Given the stochastic nature of LLM predictions and the small differences in some settings (e.g., the 10-day Overall setting under Qwen3-35B-A3B where ToE is marginally worse than Pipe), the 41.24% average improvement is not statistically supported. Additionally, the released package does not include the raw news corpus and market-price files, so an independent researcher cannot reproduce the reported numbers without obtaining the (unidentified) data source. The authors should make the data available (or provide a public benchmark link) and report per-seed results or error bars.","section":"Experimental Setup and Data availability and reproducibility"},{"comment":"The main results depend on hyperparameters eta = 10, lambda = 0.8, k1 = 8, L = 3, and q0 = 1 with no sensitivity analysis. This is especially concerning for eta = 10 combined with reliability clipping to [0,2]: in Eq. (A6), the boundary factor and denominator produce an update of approximately 10 * r_t for first-use experiences (n_hit = 0), which saturates or zeroes the reliability after a single feedback signal. That is inconsistent with the text's description of incremental and graded calibration. A sensitivity analysis (e.g., eta in {1,3,10,30} and lambda in {0.5,0.8,1.0}) is needed to confirm the results are not artifacts of these fixed values.","section":"Hyperparameters in FinEvolveBench and Formula-based experience-reliability update in FinEvolveBench"},{"comment":"FinEvolveBench is authored by the same group (Deng et al. 2026), and for this benchmark the ToE hierarchy is manually specified as industry, factor, and analysis_direction, which mirrors the benchmark's internal structure. The comparison against Mem0 and MemRL therefore includes a structural advantage for ToE that is not a property of the framework itself. The paper should either evaluate ToE on a task where the hierarchy is induced automatically (as described in Section 'Granularity of Analytical Perspectives') or include a variant of ToE with an induced hierarchy on FinEvolveBench to disentangle the contribution of the prescribed hierarchy from the experience-management mechanism.","section":"Performance on FinEvolveBench and Tree Representation and Decoupled Execution"}],"minor_comments":[{"comment":"The phrase 'applies the update rule defined in Section .' is missing a section number; please cite the specific equation (e.g., Eq. 9 or Eq. A6).","section":"Ablation Study on Reliability Update Mechanisms"},{"comment":"Given that three independent runs are performed, the paper should report standard deviations or per-seed values in addition to the averages.","section":"Tables 1 and 2"},{"comment":"The notation Q(E_t) in [0,2q0]^|E_t| is confusing because the update rule is defined per experience e; please clarify whether Q is a vector of per-experience reliabilities and how the vectorized update is computed.","section":"Eq. (9)"},{"comment":"The paper states that the experience tree is not a predefined template, but for FinEvolveBench the hierarchy is manually specified as a three-level structure; please reconcile this description with the manual construction used in the experiments.","section":"Granularity of Analytical Perspectives"},{"comment":"The paper should state the number of Game of 24 puzzles and the number of news items and industries in FinEvolveBench, as these scale details are needed to assess the evaluation.","section":"Experimental Setup"}],"recommendation":"major_revision","confidential_remarks":"The Game of 24 result appears to be a caching effect rather than evidence for the proposed framework; the authors should be asked to provide a cached baseline before resubmission. The FinEvolveBench benchmark is authored by the same group, and the raw data are not released, which raises reproducibility concerns that should be addressed in revision. The core idea has merit, but the paper's current evidence does not support the strong empirical claims in the abstract."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nHere's my read on the ToE paper. The framework — organizing agent experience as a hierarchy of analytical perspectives with reliability scores updated from outcomes — is a genuinely sensible design direction, and the paper is honest about what it does. The appendix explicitly admits that on Game of 24 the method is a deterministic, state-based specialization: analytical perspectives are canonicalized number states, reliability collapses to binary validity, and only verifier-confirmed suffixes are stored. That is a transposition table, not an experience-management abstraction, and the 20.4-point accuracy gain over a ToT baseline that searches from scratch is almost certainly the cache, not the hierarchy. The paper does not acknowledge this as a baseline confound.\n\nThe FinEvolveBench experiments are more on-target but weaker than the abstract suggests. The benchmark is from the same group, the three-level hierarchy (industry, factor, analysis_direction) mirrors the benchmark's own structure, the raw data is not released, and the paper reports no variance across the three runs it claims. Hyperparameters like eta=10 and lambda=0.8 have no sensitivity analysis. These are not fabrication, but they are enough to keep the headline '41.24% improvement' from being taken at face value.\n\nCredit where due: the framework is coherent, the writing is clear, the prompt templates are included, and the authors do not try to hide the Game of 24 specialization. The related work is adequate and the claim about conventional memory methods underperforming experience-free baselines is interesting and plausibly worth testing. Same-group benchmarks are common, but the structural fit between method and benchmark should have been addressed head-on.\n\nWho is this for? Researchers working on agent memory and continual learning will find the ToE design a useful reference, especially the reliability-update rule and the hierarchical retrieval idea. But as evidence for the central claim, the paper is not there yet.\n\nMy recommendation: send it to peer review, but with the expectation of major revision. A serious referee can push for stronger evaluation: a ToT baseline equipped with the same verified-suffix cache, a benchmark not designed by the same group, variance/error bars, and sensitivity analyses for the free parameters. If those are supplied, the paper could be a solid contribution. If not, the empirical claims should not stand.","headline":"A coherent framework with an honest appendix that reveals the Game of 24 result is mostly caching; FinEvolveBench needs independent evaluation before the headline claims are credible.","tokens_in":17740,"tokens_out":2476,"would_cite":true,"duration_ms":22170,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"An LLM agent that stores experience as a tree of reliability-weighted reasoning paths evolves faster and solves hard tasks better.","keywords":["Tree-of-Experience","experience management","self-evolving agents","hierarchical reasoning","large language models","reliability calibration","outcome-level feedback","Tree of Thoughts"],"falsifier":"Two concrete checks settle the claim. On Game of 24, run the ToT baseline with the same verifier-confirmed solution-suffix cache that ToE uses (exact canonical-state memoization, with or without reliability weighting) and see whether the 85.3% versus 64.9% gap persists or collapses; the paper's appendix says failed continuations are excluded by design, which makes this comparison the crux. On FinEvolveBench, give the experience-free Pipe the same three-level industry–factor–analysis_direction decomposition without reliability-weighted retrieval, and also try inducing the hierarchy from reasoning traces by clustering instead of specifying it by hand; if the 41.24% average tsIC gain shrinks materially in either variant, the advantage is the pre-built structure rather than the calibration mechanism.","tokens_in":16746,"feed_emoji":"🌳","tokens_out":13634,"duration_ms":110804,"temperature":0.7,"pith_summary":"This paper claims that how an LLM agent stores experience matters as much as what it stores, and proposes Tree-of-Experience (ToE). ToE keeps past analytical perspectives in a shared tree where every root-to-leaf path is a reasoning strategy carrying a reliability score that environmental feedback raises or lowers, so an outcome can be credited to the exact perspectives that produced it. On Game of 24, ToE reaches 85.3% accuracy versus 64.9% for the experience-free Tree-of-Thought baseline and cuts average LLM calls from 51.7 to 11.0; on FinEvolveBench, it improves the time-series information coefficient by 41.24% on average across 12 settings, while flat-memory baselines often do worse than no memory at all. If right, this points to memory as a structured reasoning resource: an agent that mirrors its own reasoning in its memory could reuse how to analyze a problem instead of matching whole past problems.","feed_headline":"Tree of past reasoning lifts LLM accuracy 20 points","feed_subtitle":"Reusing verified reasoning paths by perspective cuts LLM calls 78.7% and beats flat-memory baselines.","key_machinery":"The central object is the experience tree $\\mathcal{E} = \\{(\\Pi_i, Q_i, M_i)\\}$: each node is an analytical perspective (in Game of 24, a canonicalized set of remaining numbers; in FinEvolveBench, an industry, a factor, or an analysis direction), each root-to-leaf path $\\Pi_i$ is a full reasoning strategy, and each path carries a reliability state $Q_i$ in $[0, 2q_0]$ that environmental feedback raises or lowers after every use. The tree carries the whole framework: retrieval reranks the children of the current reasoning node and keeps the top-$k$ perspectives; a proposer invents new perspectives when none fit, subject to semantic subsumption, discriminative value, and balanced granularity; and maintenance merges similar nodes by embedding search plus an LLM judge, while reliability decay provides soft forgetting. The load-bearing update rule for stochastic environments is $Q(E_t) \\leftarrow Q(E_t) + \\eta \\frac{1 - ((Q(E_t)-q_0)/q_0)^2}{1 + \\log(1 + n_{\\mathrm{hit}})} \\cdot r_t$, which collapses to the binary rule $Q(e) = r_t \\in \\{0,1\\}$ on Game of 24, where a deterministic verifier decides success after a single observation.","core_discovery":"The paper's central claim is that experience representations fail when they are disconnected from the reasoning process they are meant to aid, and that organizing experience as a hierarchy of analytical perspectives fixes this. In ToE, each stored experience is a root-to-leaf reasoning path whose reliability is tuned by outcome-level feedback, so a delayed success or failure can be propagated to the specific perspectives that produced it. The authors argue that this yields the four properties they set out: attributability of outcomes to reasoning components, transferability by reusing partial reasoning paths rather than whole task instances, evolvability through incremental calibration and merging of nodes, and efficiency through branch-limited retrieval. Empirically, on Game of 24 ToE reaches 85.3% accuracy against 64.9% for Tree-of-Thoughts while cutting LLM calls by 78.7%, and on FinEvolveBench it raises tsIC by an average of 41.24% across 12 evaluation settings, in a regime where conventional experience methods (Mem0, MemRL, ReMe) often underperform the experience-free pipeline. The paper also argues that traditional intra-trajectory and inter-trajectory experience methods fail specifically because they cannot attribute noisy outcome feedback, which can actively harm performance on low-repetition tasks.","pith_inferences":["My reading: the Game of 24 result is largely a solved-suffix cache effect, because ToE stores verifier-confirmed continuations keyed by canonical number states, so any repeated or overlapping state short-circuits search. Equipping the ToT baseline with the same verified-suffix memoization would likely close much of the 20.4-point gap, and what remains would isolate the value of hierarchical attrib","Because the FinEvolveBench hierarchy (industry to factor to analysis_direction) is manually specified to mirror the benchmark's own structure, a decisive test of the transfer claim would give the experience-free pipeline the same three-level decomposition without reliability-weighted retrieval, or learn the hierarchy from reasoning traces by clustering, as the paper says the initial hierarchy can ","The reliability scores suggest a natural next step the paper does not take: treating them as a cost-sensitive signal for deciding when to retrieve versus when to explore by proposing new perspectives, which would make the tree's growth itself adaptive to feedback.","The 78.7% call reduction hints that ToE could serve as a general retrieval front-end for other structured-reasoning methods beyond the two tasks tested, though that generalization is my extrapolation, not the paper's claim."],"forward_implications":["On tasks with a well-defined reasoning structure and deterministic feedback (Game of 24), ToE raises accuracy from 64.9% to 85.3% over the experience-free ToT baseline and cuts average LLM calls per puzzle from 51.7 to 11.0, a 78.7% reduction.","On low-repetition tasks with delayed, implicit outcome feedback (FinEvolveBench), ToE improves tsIC by 41.24% on average across 12 backbone–horizon–phase settings, with larger gains in the exploitation phase once the tree has stabilized.","Conventional experience-management methods (Mem0, MemRL, ReMe) frequently underperform the experience-free baselines on both benchmarks, which the paper offers as evidence that mis-attributed experience can actively degrade reasoning.","The framework is task-agnostic in structure: the same tree-plus-reliability machinery instantiates as verifier-confirmed state caching in Game of 24 and as a three-level industry–factor–analysis-direction hierarchy in financial sentiment.","The formula-based reliability update outperforms an LLM-based judge in an ablation, suggesting that constrained, interpretable credit assignment beats free-form reflection when feedback is noisy."],"supporting_citations":[{"why":"Supplies the Tree-of-Thoughts formulation, the Game of 24 task, and the experience-free ToT baseline that ToE must beat.","marker":"Yao et al. 2023"},{"why":"Provides FinEvolveBench, the benchmark-provided Pipe baseline, and the tsIC/csIC metrics on which ToE reports its main gains.","marker":"Deng et al. 2026"},{"why":"Defines the intra-trajectory reflection paradigm that ToE argues cannot attribute outcome feedback to specific reasoning components.","marker":"Shinn et al. 2023"},{"why":"MemRL, a compared experience-management method that underperforms the experience-free baseline on both benchmarks.","marker":"Zhang et al. 2026b"},{"why":"ReMe, a compared procedural-memory method on Game of 24, excluded from the financial setting because of token overhead.","marker":"Cao et al. 2026"},{"why":"Mem0, a compared graph-memory baseline that ToE outperforms on both tasks.","marker":"Chhikara et al. 2025"},{"why":"Chain-of-Thought, the structured-reasoning premise behind the claim that memory should mirror progressive reasoning.","marker":"Wei et al. 2022"}],"fun_headline_variants":["Tree-of-Experience boosts LLM accuracy 31.4%","LLM agents evolve faster with tree-structured memory","Hierarchical reasoning memory cuts LLM calls 78.7%","Reuse reasoning paths, not whole tasks, for LLM learning"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The reported gains assume that the experience-free baselines were not given any comparable structured memory, and on both benchmarks ToE enjoys a structural head start: in Game of 24 it reuses verifier-confirmed solution suffixes that ToT must rediscover from scratch, and in FinEvolveBench it is given a hand-built industry–factor–analysis hierarchy that matches the benchmark while the plain pipeline lacks any hierarchy at all.","fun_headline_variants_meta":{"raw":{"variants":["Tree-of-Experience boosts LLM accuracy 31.4%","LLM agents evolve faster with tree-structured memory","Hierarchical reasoning memory cuts LLM calls 78.7%","Reuse reasoning paths, not whole tasks, for LLM learning"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000182,"raw_usage":{"total_tokens":1361,"prompt_tokens":1044,"completion_tokens":317,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":660,"completion_tokens_details":{"reasoning_tokens":246}},"tokens_in":660,"tokens_out":317,"duration_ms":3660,"temperature":1.0,"reasoning_tokens":246,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T04:16:28.490189+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Two concrete checks settle the claim. On Game of 24, run the ToT baseline with the same verifier-confirmed solution-suffix cache that ToE uses (exact canonical-state memoization, with or without reliability weighting) and see whether the 85.3% versus 64.9% gap persists or collapses; the paper's appendix says failed continuations are excluded by design, which makes this comparison the crux. On FinEvolveBench, give the experience-free Pipe the same three-level industry–factor–analysis_direction decomposition without reliability-weighted retrieval, and also try inducing the hierarchy from reasoning traces by clustering instead of specifying it by hand; if the 41.24% average tsIC gain shrinks materially in either variant, the advantage is the pre-built structure rather than the calibration mechanism.","supporting_citations":[],"review_version":1}