{"id":"d45c95d2-b5b3-4356-93a3-4d941a71bc00","arxiv_id":"2511.20297","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"An LLM agent can improve by storing and Monte-Carlo-refining concept-level natural-language recipes mined from its own past trajectories.","lead":"BREW builds a searchable knowledge base of natural-language how-to recipes from an LLM agent's past screen and tool interactions, then uses a Monte Carlo search to refine those recipes. On three agent benchmarks it reports 4-8% relative gains over the base agent, not the 10-20% claimed in the abstract.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Appendix A.5's 'All numbers are reported on the remaining train set' makes the final evaluation ambiguous: Table 1 may not be an independent test of BREW's claims.","rationale":"The reader's weakest-assumption analysis identifies the same load-bearing concern: the reward-evaluation queries in EG-MCTS may overlap with or inadequately represent the final evaluation set, and Appendix A.5's wording ('All numbers are reported on the remaining train set') is ambiguous at best and damning at worst. My read of the full text reinforces this. Algorithm 5 explicitly uses Q_eval to score every candidate document state, and Section 3.4 defines the correctness reward over a 'held-out query set Q.' With only ~10 queries per benchmark described in Appendix A.5, the search's reward signal is extremely noisy, and no seeds are reported to account for variance. Additionally, the paper's own ablation in Appendix B.1 (Table 5) shows that increasing MCTS depth from 3 to 10 makes OSWorld performance drop from 47.56 to 43.83, which the authors attribute to over-optimization on the train set. This makes the absence of a clear, disjoint split a serious correctness risk rather than a stylistic issue. I do not see a different, stronger flaw that supersedes this one: the method itself is plausible, the architecture is described in detail, and the qualitative analyses are useful. The issue is specifically that the empirical support for the headline quantitative claim has an unresolved evaluation-protocol ambiguity. That justifies retaining the CONDITIONAL verdict, not full acceptance. The reader's suggested request for explicit split details, multiple seeds, and error bars is the right remedy; I am agreeing with that assessment rather than proposing a different verdict.","tokens_in":22873,"tokens_out":2914,"duration_ms":32765,"concrete_test":"Ask the authors to release the exact query identifiers for (a) the EG-MCTS reward-evaluation set Q_eval used in Algorithm 5/Appendix A.5 for each benchmark, and (b) the final evaluation set behind Table 1. Then rerun BREW-MCTS with a strictly disjoint split: training trajectories for KB generation, a validation set limited to reward computation, and a separate test set (at least ~30 queries per benchmark, with 3 independent seeds). If any final-evaluation query appears in Q_eval, or if the 10–20% gains shrink below baseline variance under the disjoint split, the central claim is unsupported. If the gains reproduce on a clean held-out test set, the ambiguity is resolved as a typo and the paper's main result stands.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that BREW yields 10–20% task-success gains depends on the final evaluation being a fair, held-out test of the KB produced by EG-MCTS. The paper's own split description undermines this condition. In Appendix A.5, after describing training data (20–30 trajectories) and eval queries (10 per benchmark), the text says: 'All numbers are reported on the remaining train set.' If taken literally, Table 1 reports performance on the same distribution used to compute the reward in Eq. (7) during EG-MCTS, so the reported numbers are not an independent estimate of generalization. If it is a typo for 'test set,' the reward is still computed on only ~10 queries per benchmark (Algorithm 5's Q_eval), with no error bars or multiple seeds, making overfitting to that small reward set a real risk. This is not a hypothetical concern: the paper's own appendix B.1/Table 5 shows that increasing MCTS depth, which the authors describe as 'over optimizes on the train set,' reduces OSWorld performance from 47.56 to 43.83—evidence that the search can overfit the reward-evaluation distribution. Because the same reward-eval queries may be leaked into final numbers, and because the core contribution is the MCTS reward-guided optimization, this ambiguity is the load-bearing weak point of the empirical claim.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces BREW, a framework that converts an LLM agent's past interaction trajectories into a structured knowledge base of concept-level natural-language documents. A Reflector agent extracts concepts and insights from rollouts; an Integrator agent maintains per-concept documents; and a novel Expand-and-Gather MCTS (EG-MCTS) algorithm searches over document states, using rewards that combine task correctness (Eq. 7) and retrieval MRR (Eq. 8). The authors evaluate BREW on OSWorld, tau2-Bench, and SpreadsheetBench, comparing against a baseline and two memory baselines (Cognee, Agent-Mem), and report improvements in task success and execution efficiency. The abstract and introduction claim 10–20% gains in task precision and 10–15% fewer steps, while Table 1 shows smaller relative gains (roughly 4–8%). The paper's main contribution is the formulation of KB construction as a reward-guided state-space search over modular, concept-localized documents.","tokens_in":23267,"tokens_out":4215,"duration_ms":47178,"significance":"If the empirical claims hold, BREW is a useful and original step toward interpretable, memory-augmented agent optimization. The state-space formulation of KB construction is well-motivated, the modular concept-level document structure is a sensible alternative to flat prompt memory, and the inclusion of both correctness and retrieval rewards is a thoughtful design. The paper also provides pseudocode and qualitative KB examples, which help reproducibility. However, the contribution is empirical: the value rests on the claim that EG-MCTS produces a KB that improves held-out task performance. The evaluation as written does not yet support that claim at the advertised magnitude, because of ambiguity in the train/eval/test split, lack of repeated runs/error bars, and overbroad statements in the abstract and Section 1.","major_comments":[{"comment":"The sentence 'All numbers are reported on the remaining train set' undermines the central empirical claim. Since Eq. (7) and Algorithm 5 compute rewards on Q_eval, and EG-MCTS optimizes KB states against those rewards, the final numbers in Table 1 must be on a disjoint test set. If the sentence is literal, Table 1 is not an independent evaluation and the main conclusion is unsupported. If it is a typo for 'test set,' the reward is estimated on only ~10 queries per benchmark, and Table 5 provides direct evidence of overfitting to the reward set: increasing MCTS depth decreases OSWorld from 47.56 to 43.83, which the authors describe as over-optimizing on the train set. Please specify the exact split, report test-set numbers, and include multiple seeds or bootstrap confidence intervals.","section":"Appendix A.5"},{"comment":"The abstract and introduction claim '10–20% gains in task success' and '10–15% fewer execution steps.' These figures are not supported by Table 1 or Section 5.2. Relative gains in Table 1 are: OSWorld 47.56/44.20 − 1 ≈ 7.6%, tau2-Bench 59.14/56.63 − 1 ≈ 4.4%, and SpreadsheetBench 46.80/44.30 − 1 ≈ 5.6%. For efficiency, OSWorld improves from ~75 to ~64 steps (≈14.7%), tau2 turns decrease by ~3.5% (29.47 to 28.43), and SpreadsheetBench turns increase from 4.5 to 5.4. Please replace the current global claims with the actual per-benchmark relative improvements, or justify a different aggregation.","section":"Abstract and Section 1"},{"comment":"All results are single point estimates. The differences between BREW-MCTS and the best alternative are small: tau2-Bench shows BREW-Greedy and BREW-MCTS both at 59.14; SpreadsheetBench has 46.80 (MCTS) vs 45.94 (Greedy) vs 44.30 (baseline); OSWorld has 47.56 (MCTS) vs 46.70 (Cognee). With reward optimization on ~10 eval queries and no error bars or multiple seeds, it is not possible to assess whether these differences are meaningful. Please report variance across seeds or at least bootstrap confidence intervals over tasks.","section":"Table 1 and Section 5"},{"comment":"The paper claims 'consistently outperforming existing memory-augmented baselines,' but Table 1 includes only Cognee and Agent-Mem. The related work discusses MetaReflection and GEPA as directly relevant prompt/memory optimization methods, yet they are not included as baselines. Given the paper's own framing and the central claim of state-of-the-art results, the comparison set is too narrow. Please add these baselines, or explicitly restrict the claim to the tested baselines and remove 'state-of-the-art' phrasing.","section":"Section 2 and Table 1"}],"minor_comments":[{"comment":"Step-Back Prompting and In-Context Learning are listed as baseline methods, but no results for them appear in Table 1. Either add their results or remove the description.","section":"Appendix A.3"},{"comment":"The notation 'D_t = d_t ∪ d^*_{i:i≠t}' is ambiguous. It should state that d_t is the candidate document for concept k and that the union is over all other concepts' best documents, with explicit index ranges. As written, the expression mixes a document with a set and is confusing.","section":"Section 3.3, Eq. (5)"},{"comment":"The text says the baseline averages ~75 steps and BREW ~64 steps, while Figure 3's y-axis is 'Average Step Reduction (Successful Cases)' with values 14–23. Clarify whether the 75→64 numbers are over all tasks or only successful cases, and ensure the figure and text use the same definition.","section":"Figure 3 and Section 5.2"},{"comment":"There are typos: 'texttto4-mini' should be 'o4-mini', and 'SPREADSHEET ENCH' should be 'SpreadsheetBench' in the retrieval-strategy paragraph. Also standardize the spelling of 'tau2-Bench' across the paper.","section":"Appendix A.4.3"}],"recommendation":"major_revision","confidential_remarks":"The ambiguous split sentence in Appendix A.5 is likely a typo, but it must be corrected because the entire empirical contribution depends on a clean train/eval/test separation. Even after that correction, the paper needs variance estimates and a more careful statement of the gains actually observed. I am not recommending rejection because the method is original and the qualitative analysis is compelling; however, the claims in their current form exceed the evidence."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: BREW is a sensible, well-scoped idea—distill trajectories into concept-level KB documents and use reward-guided MCTS to refine them—and the qualitative material is the strongest part. But the empirical case is weaker than the abstract suggests, and the evaluation split is ambiguous enough to be the load-bearing flaw.\n\nWhat is new: the parallel per-concept EG-MCTS with correctness and retrieval rewards is a reasonable way to search over memory states, and it goes beyond the cited self-referential prior work (MetaReflection, StackFeed). The paper also ships a clear architecture, compares against iterative and greedy search, and gives detailed, concrete SpreadsheetBench examples (header detection, ISBLANK vs ==\"\", range compliance) that look like genuine knowledge transfer rather than prompt artifacts.\n\nWhere it gets soft: Appendix A.5 says \"All numbers are reported on the remaining train set\" after defining a 20-30 trajectory training set and 10 eval queries. If that is literal, Table 1 is not an independent test, and the whole generalization claim collapses. If it's a typo for \"test set,\" the rewards in Eq. (7) are computed on only ~10 queries per benchmark, with no seeds or error bars, and B.1 shows the search can overfit that reward distribution (OSWorld drops from 47.56 to 43.83 with more depth). Either way the central table is not enough. On top of that, the abstract promises 10-20% gains and 10-15% fewer steps, while Table 1 shows relative success gains of 4-8% and step reductions only on OSWorld; SpreadsheetBench turns go up. The self-citation pattern (GEPA, MetaReflection, StackFeed) is not itself a flaw, but there is no ablation that removes the search and just dumps all insights into a flat KB, so the MCTS's contribution is not pinned down.\n\nWho this is for: people working on memory-augmented agents and in-context optimization; the paper asks the right question. It deserves a serious referee—not a desk reject—but the referee should require a corrected split description, error bars across seeds, code, and that ablation. If the train-set wording is literal, that's a rejectable flaw; if it's a typo, it's still a weak evidence base that needs revision. My own verdict: promising direction, unproven numbers.","headline":"BREW's memory-search idea is plausible and the qualitative evidence is good, but the final evaluation may not be independent of the training reward and the abstract overstates the gains.","tokens_in":23737,"tokens_out":3345,"would_cite":false,"duration_ms":37470,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"BREW claims agents can learn from experience by building a searchable knowledge base of concept-level recipes, improving task success by 10-20% and cutting tool calls by 10-15%.","keywords":["language agents","experiential knowledge","knowledge base construction","Monte Carlo tree search","memory augmentation","retrieval","procedural recipes","LLM agents"],"falsifier":"Re-run the full BREW pipeline on the three benchmarks with a genuinely disjoint held-out test set and at least five random seeds, building the KB only from the designated training queries; if the 10–20% precision gain and 10–15% step reduction do not reproduce on that independent set, the central claim fails. A simpler probe: remove the retrieval reward (set λ_ret=0) and check whether task success degrades; if it does not, the joint-optimization story is unsupported.","tokens_in":22818,"feed_emoji":"🧠","tokens_out":5660,"duration_ms":54366,"temperature":0.7,"pith_summary":"This paper argues that large language model agents need not be fine-tuned to improve: they can instead learn from their own past interactions by distilling trajectories into a structured, natural-language knowledge base. The proposed framework, BREW, partitions memory into concept-level documents—recipes that say what to do, when it applies, and what to watch out for—and treats the selection and refinement of those documents as a state-space search. A reward-guided Monte Carlo tree search, EG-MCTS, jointly optimizes each document's contribution to task correctness and its retrievability. On three real-world benchmarks—computer-use automation, tool-use conversation, and spreadsheet manipulation—the authors report 10–20% gains in task success and 10–15% fewer execution steps over base agents, with an inspectable, modular knowledge base as a side benefit. A sympathetic reader would care because this offers a transparent, low-cost alternative to weight optimization for making agents improve across sessions.","feed_headline":"LLM agents gain 10-20% success from learned recipe memory","feed_subtitle":"A searchable KB of concept-level recipes, refined by reward-guided MCTS, also cuts tool calls by 10-15%.","key_machinery":"The load-bearing mechanism is Expand-and-Gather Monte Carlo Tree Search (EG-MCTS), an algorithm that searches the space of possible knowledge bases. The space is factored into independent per-concept trees, where each node is a candidate natural-language document for that concept. At each iteration, every tree selects its best node by UCT, expands it into several new candidate documents, and then gathers the current best nodes from all trees to form a hybrid KB used for reward estimation. Rewards are a weighted combination of a correctness score (evaluating the agent's task success on held-out queries) and a retrieval score (mean reciprocal rank of the document within the KB). This design le","core_discovery":"At the core of the paper is the claim that an agent's memory can be organized as a collection of concept-localized documents in natural language, and that the best such collection can be found by search rather than by hand-authoring or prompt-stuffing. Each document encodes a reusable procedure discovered from trajectories—e.g., how to search and open files, how to handle blank cells in Excel, when to deny an unsupported request—and is annotated with conditions of applicability and failure warnings. The authors formalize KB construction as a search over document states, and introduce Expand-and-Gather MCTS: per-concept search trees are expanded in parallel, but rewards are computed against a","pith_inferences":["If the transfer assumption holds, a natural extension is to start from a very small seed of trajectories in a new domain and let BREW grow the KB online, potentially reducing the data needed to specialize an agent.","The same formulation—searching document states with correctness and retrieval rewards—could apply to optimizing other retrievable artifacts, such as API documentation or instruction sets, where retrieval quality matters.","A testable extension would be to decouple the two rewards and measure their individual contributions; the paper reports only the balanced setting (λ=0.5), so the interaction is not isolated.","The evaluation's reliance on a tiny held-out reward set (about 10 queries per benchmark) means the reported numbers should be re-checked on a larger, truly disjoint test set before treating the 10–20% gains as robust; this is our caution, not the paper's."],"forward_implications":["If correct, agents can be improved across sessions by updating a text knowledge base, without retraining model weights, making improvement both cheaper and inspectable.","Because the KB is modular and concept-level, users can read, edit, or extend what the agent knows—offering a control surface for steering behavior in deployed systems.","The reported gains concentrate in tasks with shared sub-trajectories (e.g., UI automation), implying that experience distilled at the concept level transfers across similar tasks within a domain.","Even where success rates do not rise, the authors observe 14–23 fewer execution steps on OSWorld successes, indicating efficiency gains from memory even without accuracy gains.","The failure of existing memory-augmented baselines to beat a memoryless agent suggests that naive memory accumulation can hurt; BREW's search over memory states is what avoids such degradation."],"fun_headline_variants":["BREW: agents learn from experience, boosting success 10-20%","Agent memory of past tricks lifts success 10-20%","BREW's recipe book: 10-20% better agents from experience","Experiential recipes cut steps and boost agent success 10-20%"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The load-bearing premise is that the roughly ten held-out queries per benchmark used to compute EG-MCTS rewards are representative enough that optimizing document states on them transfers to the final evaluation set—and, as the paper's appendix ambiguously states 'All numbers are reported on the remaining train set,' it is not fully clear the final numbers come from an independent test set.","fun_headline_variants_meta":{"raw":{"variants":["BREW: agents learn from experience, boosting success 10-20%","Agent memory of past tricks lifts success 10-20%","BREW's recipe book: 10-20% better agents from experience","Experiential recipes cut steps and boost agent success 10-20%"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000805,"raw_usage":{"total_tokens":3411,"prompt_tokens":820,"completion_tokens":2591,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":564,"completion_tokens_details":{"reasoning_tokens":2512}},"tokens_in":564,"tokens_out":2591,"duration_ms":19794,"temperature":1.0,"reasoning_tokens":2512,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-03T20:16:44.788719+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the full BREW pipeline on the three benchmarks with a genuinely disjoint held-out test set and at least five random seeds, building the KB only from the designated training queries; if the 10–20% precision gain and 10–15% step reduction do not reproduce on that independent set, the central claim fails. A simpler probe: remove the retrieval reward (set λ_ret=0) and check whether task success degrades; if it does not, the joint-optimization story is unsupported.","supporting_citations":[],"review_version":1}