Pith. sign in

REVIEW 4 major objections 4 minor 84 references

BREW claims agents can learn from experience by building a searchable knowledge base of concept-level recipes, improving task success by 10-20% and cutting tool calls by 10-15%.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 20:16 UTC pith:XPIFDTAD

load-bearing objection BREW's memory-search idea is plausible and the qualitative evidence is good, but the final evaluation may not be independent of the training reward and the abstract overstates the gains. the 4 major comments →

arxiv 2511.20297 v2 pith:XPIFDTAD submitted 2025-11-25 cs.AI

Improving Language Agents through BREW: Bootstrapping expeRientially-learned Environmental knoWledge

Shashank Kirtania , Param Biyani , Priyanshu Gupta , Yasharth Bajpai , Roshni Iyer , Sumit Gulwani , Gustavo Soares This is my paper
classification cs.AI
keywords language agentsexperiential knowledgeknowledge base constructionMonte Carlo tree searchmemory augmentationretrievalprocedural recipesLLM agents
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper argues that large language model agents need not be fine-tuned to improve: they can instead learn from their own past interactions by distilling trajectories into a structured, natural-language knowledge base. The proposed framework, BREW, partitions memory into concept-level documents—recipes that say what to do, when it applies, and what to watch out for—and treats the selection and refinement of those documents as a state-space search. A reward-guided Monte Carlo tree search, EG-MCTS, jointly optimizes each document's contribution to task correctness and its retrievability. On three real-world benchmarks—computer-use automation, tool-use conversation, and spreadsheet manipulation—the authors report 10–20% gains in task success and 10–15% fewer execution steps over base agents, with an inspectable, modular knowledge base as a side benefit. A sympathetic reader would care because this offers a transparent, low-cost alternative to weight optimization for making agents improve across sessions.

Core claim

At the core of the paper is the claim that an agent's memory can be organized as a collection of concept-localized documents in natural language, and that the best such collection can be found by search rather than by hand-authoring or prompt-stuffing. Each document encodes a reusable procedure discovered from trajectories—e.g., how to search and open files, how to handle blank cells in Excel, when to deny an unsupported request—and is annotated with conditions of applicability and failure warnings. The authors formalize KB construction as a search over document states, and introduce Expand-and-Gather MCTS: per-concept search trees are expanded in parallel, but rewards are computed against a

What carries the argument

The load-bearing mechanism is Expand-and-Gather Monte Carlo Tree Search (EG-MCTS), an algorithm that searches the space of possible knowledge bases. The space is factored into independent per-concept trees, where each node is a candidate natural-language document for that concept. At each iteration, every tree selects its best node by UCT, expands it into several new candidate documents, and then gathers the current best nodes from all trees to form a hybrid KB used for reward estimation. Rewards are a weighted combination of a correctness score (evaluating the agent's task success on held-out queries) and a retrieval score (mean reciprocal rank of the document within the KB). This design le

Load-bearing premise

The load-bearing premise is that the roughly ten held-out queries per benchmark used to compute EG-MCTS rewards are representative enough that optimizing document states on them transfers to the final evaluation set—and, as the paper's appendix ambiguously states 'All numbers are reported on the remaining train set,' it is not fully clear the final numbers come from an independent test set.

What would settle it

Re-run the full BREW pipeline on the three benchmarks with a genuinely disjoint held-out test set and at least five random seeds, building the KB only from the designated training queries; if the 10–20% precision gain and 10–15% step reduction do not reproduce on that independent set, the central claim fails. A simpler probe: remove the retrieval reward (set λ_ret=0) and check whether task success degrades; if it does not, the joint-optimization story is unsupported.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • If correct, agents can be improved across sessions by updating a text knowledge base, without retraining model weights, making improvement both cheaper and inspectable.
  • Because the KB is modular and concept-level, users can read, edit, or extend what the agent knows—offering a control surface for steering behavior in deployed systems.
  • The reported gains concentrate in tasks with shared sub-trajectories (e.g., UI automation), implying that experience distilled at the concept level transfers across similar tasks within a domain.
  • Even where success rates do not rise, the authors observe 14–23 fewer execution steps on OSWorld successes, indicating efficiency gains from memory even without accuracy gains.
  • The failure of existing memory-augmented baselines to beat a memoryless agent suggests that naive memory accumulation can hurt; BREW's search over memory states is what avoids such degradation.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the transfer assumption holds, a natural extension is to start from a very small seed of trajectories in a new domain and let BREW grow the KB online, potentially reducing the data needed to specialize an agent.
  • The same formulation—searching document states with correctness and retrieval rewards—could apply to optimizing other retrievable artifacts, such as API documentation or instruction sets, where retrieval quality matters.
  • A testable extension would be to decouple the two rewards and measure their individual contributions; the paper reports only the balanced setting (λ=0.5), so the interaction is not isolated.
  • The evaluation's reliance on a tiny held-out reward set (about 10 queries per benchmark) means the reported numbers should be re-checked on a larger, truly disjoint test set before treating the 10–20% gains as robust; this is our caution, not the paper's.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper introduces BREW, a framework that converts an LLM agent's past interaction trajectories into a structured knowledge base of concept-level natural-language documents. A Reflector agent extracts concepts and insights from rollouts; an Integrator agent maintains per-concept documents; and a novel Expand-and-Gather MCTS (EG-MCTS) algorithm searches over document states, using rewards that combine task correctness (Eq. 7) and retrieval MRR (Eq. 8). The authors evaluate BREW on OSWorld, tau2-Bench, and SpreadsheetBench, comparing against a baseline and two memory baselines (Cognee, Agent-Mem), and report improvements in task success and execution efficiency. The abstract and introduction claim 10–20% gains in task precision and 10–15% fewer steps, while Table 1 shows smaller relative gains (roughly 4–8%). The paper's main contribution is the formulation of KB construction as a reward-guided state-space search over modular, concept-localized documents.

Significance. If the empirical claims hold, BREW is a useful and original step toward interpretable, memory-augmented agent optimization. The state-space formulation of KB construction is well-motivated, the modular concept-level document structure is a sensible alternative to flat prompt memory, and the inclusion of both correctness and retrieval rewards is a thoughtful design. The paper also provides pseudocode and qualitative KB examples, which help reproducibility. However, the contribution is empirical: the value rests on the claim that EG-MCTS produces a KB that improves held-out task performance. The evaluation as written does not yet support that claim at the advertised magnitude, because of ambiguity in the train/eval/test split, lack of repeated runs/error bars, and overbroad statements in the abstract and Section 1.

major comments (4)
  1. [Appendix A.5] The sentence 'All numbers are reported on the remaining train set' undermines the central empirical claim. Since Eq. (7) and Algorithm 5 compute rewards on Q_eval, and EG-MCTS optimizes KB states against those rewards, the final numbers in Table 1 must be on a disjoint test set. If the sentence is literal, Table 1 is not an independent evaluation and the main conclusion is unsupported. If it is a typo for 'test set,' the reward is estimated on only ~10 queries per benchmark, and Table 5 provides direct evidence of overfitting to the reward set: increasing MCTS depth decreases OSWorld from 47.56 to 43.83, which the authors describe as over-optimizing on the train set. Please specify the exact split, report test-set numbers, and include multiple seeds or bootstrap confidence intervals.
  2. [Abstract and Section 1] The abstract and introduction claim '10–20% gains in task success' and '10–15% fewer execution steps.' These figures are not supported by Table 1 or Section 5.2. Relative gains in Table 1 are: OSWorld 47.56/44.20 − 1 ≈ 7.6%, tau2-Bench 59.14/56.63 − 1 ≈ 4.4%, and SpreadsheetBench 46.80/44.30 − 1 ≈ 5.6%. For efficiency, OSWorld improves from ~75 to ~64 steps (≈14.7%), tau2 turns decrease by ~3.5% (29.47 to 28.43), and SpreadsheetBench turns increase from 4.5 to 5.4. Please replace the current global claims with the actual per-benchmark relative improvements, or justify a different aggregation.
  3. [Table 1 and Section 5] All results are single point estimates. The differences between BREW-MCTS and the best alternative are small: tau2-Bench shows BREW-Greedy and BREW-MCTS both at 59.14; SpreadsheetBench has 46.80 (MCTS) vs 45.94 (Greedy) vs 44.30 (baseline); OSWorld has 47.56 (MCTS) vs 46.70 (Cognee). With reward optimization on ~10 eval queries and no error bars or multiple seeds, it is not possible to assess whether these differences are meaningful. Please report variance across seeds or at least bootstrap confidence intervals over tasks.
  4. [Section 2 and Table 1] The paper claims 'consistently outperforming existing memory-augmented baselines,' but Table 1 includes only Cognee and Agent-Mem. The related work discusses MetaReflection and GEPA as directly relevant prompt/memory optimization methods, yet they are not included as baselines. Given the paper's own framing and the central claim of state-of-the-art results, the comparison set is too narrow. Please add these baselines, or explicitly restrict the claim to the tested baselines and remove 'state-of-the-art' phrasing.
minor comments (4)
  1. [Appendix A.3] Step-Back Prompting and In-Context Learning are listed as baseline methods, but no results for them appear in Table 1. Either add their results or remove the description.
  2. [Section 3.3, Eq. (5)] The notation 'D_t = d_t ∪ d^*_{i:i≠t}' is ambiguous. It should state that d_t is the candidate document for concept k and that the union is over all other concepts' best documents, with explicit index ranges. As written, the expression mixes a document with a set and is confusing.
  3. [Figure 3 and Section 5.2] The text says the baseline averages ~75 steps and BREW ~64 steps, while Figure 3's y-axis is 'Average Step Reduction (Successful Cases)' with values 14–23. Clarify whether the 75→64 numbers are over all tasks or only successful cases, and ensure the figure and text use the same definition.
  4. [Appendix A.4.3] There are typos: 'texttto4-mini' should be 'o4-mini', and 'SPREADSHEET ENCH' should be 'SpreadsheetBench' in the retrieval-strategy paragraph. Also standardize the spelling of 'tau2-Bench' across the paper.

Circularity Check

1 steps flagged

Appendix A.5 reports final numbers 'on the remaining train set,' making Table 1 potentially the same data that EG-MCTS's correctness reward was optimized over.

specific steps
  1. fitted input called prediction [Appendix A.5 ('Training Data Collection'); Section 3.4, Eq. (7); Algorithm 5; Algorithm 4, line 15; Table 1]
    "OSWorld: 20 successful trajectories (2 per application domain) and 10 for evals. τ 2-Bench: 20 trajectories balanced across domains and difficulty settings and 10 for evals. SpreadsheetBench: Uniformly sample 30 trajectories for training and 10 for evaluation. All numbers are reported on the remaining train set."

    EG-MCTS scores every candidate document with EVALUATE (Algorithm 5) on the eval queries Q_eval_k (Algorithm 4, line 15: R_k,j <- EVALUATE(d_k,j, D_hybrid, Q_eval_k)), using the correctness reward of Eq. (7), and backpropagates that reward to select the final KB. The paper then reports Table 1 'on the remaining train set.' If that set includes or overlaps with the Q_eval used for rewards, the reported 10–20% gains are not independent predictions: they are the objective EG-MCTS was explicitly maximizing. Appendix B.1 shows this is not harmless — deeper MCTS 'over optimizes on the train set' and test performance drops from 47.56 to 43.83. Thus, as written, the central empirical claim is fitted input reported as prediction.

full rationale

The only circularity I can pin to a specific textual reduction is the train/eval ambiguity in Appendix A.5. If 'remaining train set' is a typo for 'test set,' Table 1 is an independent evaluation and BREW reduces to a standard, though small, validation scheme; the self-citations to StackFeed and MetaReflection are prior work but are not load-bearing because EG-MCTS is implemented and compared here. However, taken literally, the paper's own split description makes its headline result coincide with the reward set used to search the KB, so the central claim is partially circular. No other derivation step—concept deduplication, IntegAgent updates, or the Eq. (6) reward decomposition—reduces to its inputs by construction.

Axiom & Free-Parameter Ledger

7 free parameters · 5 axioms · 0 invented entities

BREW does not introduce new physical or mathematical entities. The central burden is a set of empirical assumptions about LLM extraction, retrieval, and the representativeness of small eval sets. The main free parameters are search hyperparameters and the unstated clustering threshold, which are not thoroughly ablated.

free parameters (7)
  • reward weights lambda_corr, lambda_ret = 0.5, 0.5
    Chosen by hand as balanced weights in Equation 6; no sensitivity analysis.
  • MCTS expansion width e and max depth k = e=3, k=3
    Set in Appendix A.2; Appendix B.1 shows depth/width changes alter results, so these hyperparameters matter.
  • maximum MCTS iterations n = 10
    Set in Appendix A.2; no ablation across iterations.
  • number of training trajectories = 20 for OSWorld/tau2, 30 for SpreadsheetBench
    Chosen in Appendix A.5; no scaling study except one tau2 generalizability curve.
  • concept clustering similarity threshold = unspecified
    Algorithm 3 says 'cluster concepts by similarity' but does not state the threshold or embedding model.
  • retrieval count = top-3
    Set in Appendix A.5; no ablation on retrieval count.
  • IntegAgent temperature = 0.7
    Set in Appendix A.2 to encourage diversity; no sensitivity analysis.
axioms (5)
  • domain assumption The Reflector Agent accurately extracts concept-insight pairs from trajectories and rubric labels.
    Section 3.2 defines ReflAgent as producing sentence-level insights; the paper does not evaluate extraction accuracy or the effect of extraction errors.
  • domain assumption A held-out eval set of about 10 queries per benchmark is representative for reward-guided KB search.
    Appendix A.5 states 10 queries for evals; the paper does not show stability of rewards across different eval sets.
  • domain assumption Retrieving the top-3 natural-language documents and adding them to the prompt reliably changes the agent's behavior in the intended direction.
    Section 3 and Appendix A.5 assume retrieval improves behavior; no analysis of retrieval failures or negative transfer.
  • domain assumption The task-specific grader and human-validated rubrics provide correct labels for trajectory evaluation.
    Section 3.1 relies on the correctness grader and rubrics from [6]; no error analysis of the grader is given.
  • domain assumption UCT-based MCTS with the defined reward function finds a KB state that generalizes beyond the eval set.
    Section 3.3-3.4 defines the search; the paper does not prove convergence or generalization, and Appendix B.1 shows overfitting with deeper search.

pith-pipeline@v1.3.0-alltime-deepseek · 22631 in / 10741 out tokens · 111263 ms · 2026-08-03T20:16:44.788719+00:00 · methodology

0 comments
read the original abstract

Large Language Model (LLM)-based agents are increasingly capable of complex, multi-step tasks such as GUI automation, tool use, and data manipulation, yet they cannot learn from experience: each new session rediscovers solutions from scratch. We introduce BREW (Bootstrapping expeRientially-learned Environmental knoWledge), a framework that distills an agent's past interaction trajectories into a structured, retrievable knowledge base (KB) of natural-language recipes, concept-level procedural documents that capture what to do, when it applies, and what to watch out for. Drawing on the principle of library learning from program synthesis, BREW decomposes agent memory into modular, concept-localized documents and formalizes KB construction as a state-space search problem. To navigate this space, we introduce Expand-and-Gather Monte Carlo Tree Search (EG-MCTS), a reward-guided algorithm that jointly optimizes recipe accuracy and retrievability across parallel, per-concept search trees. We further adapt hindsight relabeling to convert near-miss trajectories into positive demonstrations, surfacing latent agent competencies as reusable knowledge. On three domain-grounded benchmarks, OSWorld, tau^2-Bench, and SpreadSheetBench, BREW achieves 10-20% gains in task success and 10-15% fewer execution steps over base agents, while consistently outperforming existing memory-augmented baselines that can degrade below memoryless performance. The resulting KB is inspectable, modular, and extensible, providing a transparent and controllable substrate for agent optimization.

Figures

Figures reproduced from arXiv: 2511.20297 by Gustavo Soares, Param Biyani, Priyanshu Gupta, Roshni Iyer, Shashank Kirtania, Sumit Gulwani, Yasharth Bajpai.

Figure 1
Figure 1. Figure 1: BREW architecture overview using examples from the OSWorld dataset. Step 1 indicates the trajectory generation process with agent alignment to human-validated rubrics and correctness using task-specific grader. Steps 2–4 indicate the Reflector Agent, which learns key concepts and corresponding insights from trajectories. Step 5 indicates the Integrator Agent, which integrates knowledge from the Reflector A… view at source ↗
Figure 2
Figure 2. Figure 2: Illustration of BREW’s KB optimization process using Expand-and-Gather MCTS with OSWorld examples. In the Expand Phase, for each document k, we sample the best node from treek using UCT and perfrom node expansion. Node rewards are estimated based on correctness and retrievability. In the Gather Phase, the current best nodes from each tree are gathered at each node. The process is repeated for the next iter… view at source ↗
Figure 3
Figure 3. Figure 3: The bar plot represents the category-wise success rate over various tasks in the OSWorld [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Distribution of errors in τ 2 Bench Retail τ 2Bench. Similarly, BREW reduces average conversation turns from 29.47 to 28.43 (-3.5%), while maintaining consistent step reductions across categories. Step reductions average 1.7 steps for Retail and Telecom, but 3.1 steps for Airline, indicating greater efficiency gains in complex domains. Qualitative analysis sec￾onds these numbers showing how knowledge base … view at source ↗
Figure 5
Figure 5. Figure 5: t-SNE plot of knowledge learned by BREW (blue), an experiencally learning Algorithm, and A-mem (yellow), an agentic tool based memory storage that relying on the LLM to take memory save action, on SpreadsheetBench. The t-SNE visualization of the KB concept embeddings reveals that the knowledge learned by the BREW framework is semantically coherent, forming a dense cluster. This is in stark contrast to the … view at source ↗
Figure 6
Figure 6. Figure 6: Performance generalizability on τ 2 -Bench. The plot shows the relationship between the proportion of training data and performance on the held-out set. BREW exhibits consistent gains with increased training data, demonstrating strong generalization behavior. C Exemplar Knowledge Bases C.1 Knowledge base learned for OSWorld We showcase a small part of knowledge base learned thought BREW . This demonstrate … view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

84 extracted references · 18 linked inside Pith

  1. [1]

    Agrawal, Shangyin Tan, Dilara Soylu, Noah Ziems, Rishi Khare, Krista Opsahl-Ong, Arnav Singhvi, Herumb Shandilya, Michael J

    Lakshya A. Agrawal, Shangyin Tan, Dilara Soylu, Noah Ziems, Rishi Khare, Krista Opsahl-Ong, Arnav Singhvi, Herumb Shandilya, Michael J. Ryan, Meng Jiang, Christopher Potts, Koushik Sen, Alexandros G. Dimakis, Ion Stoica, Dan Klein, Matei Zaharia, and Omar Khattab. Gepa: Reflective prompt evolution can outperform reinforcement learning.arXiv preprint arXiv...

    Pith/arXiv arXiv 2025

  2. [2]

    Introducing computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku, Octo- ber 2024

    Anthropic. Introducing computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku, Octo- ber 2024. URL https://www.anthropic.com/news/3-5-models-and-computer-use . Accessed: 2025

    2024

  3. [3]

    Let’s fix this together: Conversational debugging with github copilot

    Yasharth Bajpai, Bhavya Chopra, Param Biyani, Cagri Aslan, Dustin Coleman, Sumit Gulwani, Chris Parnin, Arjun Radhakrishna, and Gustavo Soares. Let’s fix this together: Conversational debugging with github copilot. In2024 IEEE Symposium on Visual Languages and Human- Centric Computing (VL/HCC), pages 1–12, 2024. doi: 10.1109/VL/HCC60511.2024.00011

    arXiv 2024

  4. [5]

    τ 2-bench: Evaluating conversational agents in a dual-control environment, 2025

    Victor Barres, Honghua Dong, Soham Ray, Xujie Si, and Karthik Narasimhan. τ 2-bench: Evaluating conversational agents in a dual-control environment, 2025. URL https://arxiv. org/abs/2506.07982

    Pith/arXiv arXiv 2025

  5. [6]

    Rubicon: Rubric-based evaluation of domain-specific human ai conversations

    Param Biyani, Yasharth Bajpai, Arjun Radhakrishna, Gustavo Soares, and Sumit Gulwani. Rubicon: Rubric-based evaluation of domain-specific human ai conversations. InProceedings of the 1st ACM International Conference on AI-Powered Software, AIware 2024, page 161–169, New York, NY , USA, 2024. Association for Computing Machinery. ISBN 9798400706851. doi: 10...

    arXiv 2024

  6. [7]

    Mem0: Building production-ready ai agents with scalable long-term memory.arXiv preprint arXiv:2504.19413, 2025

    Prateek Chhikara, Dev Khant, Saket Aryan, Taranjeet Singh, and Deshraj Yadav. Mem0: Building production-ready ai agents with scalable long-term memory.arXiv preprint arXiv:2504.19413, 2025

    Pith/arXiv arXiv 2025

  7. [8]

    Plan-and-act: Improving planning of agents for long-horizon tasks.The Forty-Second International Conference on Machine Learning, 2025

    Lutfi Eren Erdogan, Nicholas Lee, Sehoon Kim, Suhong Moon, Hiroki Furuta, Gopala Anu- manchipalli, Kurt Keutzer, and Amir Gholami. Plan-and-act: Improving planning of agents for long-horizon tasks.The Forty-Second International Conference on Machine Learning, 2025

    2025

  8. [9]

    Rajamani, and Gustavo Soares

    Naman Gupta, Shashank Kirtania, Priyanshu Gupta, Krishna Kariya, Sumit Gulwani, Arun Iyer, Suresh Parthasarathy, Arjun Radhakrishna, Sriram K. Rajamani, and Gustavo Soares. Stackfeed: Structured textual actor-critic knowledge base editing with feedback, 2024. URL https://arxiv.org/abs/2410.10584

    arXiv 2024

  9. [10]

    MetaReflection: Learning instructions for language agents using past reflections

    Priyanshu Gupta, Shashank Kirtania, Ananya Singha, Sumit Gulwani, Arjun Radhakrishna, Gustavo Soares, and Sherry Shi. MetaReflection: Learning instructions for language agents using past reflections. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen, editors, Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages ...

    doi:10.18653/v1/2024.emnlp-main.477 2024

  10. [11]

    Sub- goal distillation: A method to improve small language agents, 2024

    Maryam Hashemzadeh, Elias Stengel-Eskin, Sarath Chandar, and Marc-Alexandre Cote. Sub- goal distillation: A method to improve small language agents, 2024. URL https://arxiv. org/abs/2405.02749

    Pith/arXiv arXiv 2024

  11. [12]

    Evaluating memory in llm agents via incremental multi-turn interactions, 2025

    Yuanzhe Hu, Yu Wang, and Julian McAuley. Evaluating memory in llm agents via incremental multi-turn interactions, 2025. URLhttps://arxiv.org/abs/2507.05257

    Pith/arXiv arXiv 2025

  12. [13]

    SWE-bench: Can language models resolve real-world github issues? InThe Twelfth International Conference on Learning Representations, 2024

    Carlos E Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik R Narasimhan. SWE-bench: Can language models resolve real-world github issues? InThe Twelfth International Conference on Learning Representations, 2024. URL https: //openreview.net/forum?id=VTF8yNQM66. 10

    2024

  13. [14]

    Executable functional abstractions: Inferring generative programs for advanced math problems

    Zaid Khan, Elias Stengel-Eskin, Archiki Prasad, Jaemin Cho, and Mohit Bansal. Executable functional abstractions: Inferring generative programs for advanced math problems. 2025

    2025

  14. [15]

    Bandit based monte-carlo planning

    Levente Kocsis and Csaba Szepesvári. Bandit based monte-carlo planning. In Johannes Fürnkranz, Tobias Scheffer, and Myra Spiliopoulou, editors,Machine Learning: ECML 2006, pages 282–293, Berlin, Heidelberg, 2006. Springer Berlin Heidelberg. ISBN 978-3-540-46056- 5

    2006

  15. [16]

    A review of prominent paradigms for llm-based agents: Tool use, planning (including rag), and feedback learning

    Xinzhe Li. A review of prominent paradigms for llm-based agents: Tool use, planning (including rag), and feedback learning. InProceedings of the 31st International Conference on Compu- tational Linguistics (COLING), pages 9760–9779, Abu Dhabi, UAE, 2025. Association for Computational Linguistics. URLhttps://aclanthology.org/2025.coling-main.652

    2025

  16. [17]

    Michael L. Littman. An optimization-based categorization of reinforcement learning environ- ments. 1993. URLhttps://api.semanticscholar.org/CorpusID:17988064

    1993

  17. [18]

    Alphago moment for model architecture discovery.ArXiv, abs/2507.18074, 2025

    Yixiu Liu, Yang Nan, Weixian Xu, Xiangkun Hu, Lyumanshan Ye, Zhen Qin, and Pengfei Liu. Alphago moment for model architecture discovery.ArXiv, abs/2507.18074, 2025. URL https://api.semanticscholar.org/CorpusID:280018530

    Pith/arXiv arXiv 2025

  18. [19]

    Spreadsheetbench: Towards challenging real world spreadsheet manipulation.Advances in Neural Information Processing Systems, 37:94871–94908, 2024

    Zeyao Ma, Bohan Zhang, Jing Zhang, Jifan Yu, Xiaokang Zhang, Xiaohan Zhang, Sijia Luo, Xi Wang, and Jie Tang. Spreadsheetbench: Towards challenging real world spreadsheet manipulation.Advances in Neural Information Processing Systems, 37:94871–94908, 2024

    2024

  19. [20]

    Optimizing the interface between knowledge graphs and llms for complex reasoning, 2025

    Vasilije Markovic, Lazar Obradovic, Laszlo Hajdu, and Jovan Pavlovic. Optimizing the interface between knowledge graphs and llms for complex reasoning, 2025. URL https: //arxiv.org/abs/2505.24478

    Pith/arXiv arXiv 2025

  20. [21]

    Kozlovskii, Francisco J

    Alexander Novikov, Ngân V~u, Marvin Eisenberger, Emilien Dupont, Po-Sen Huang, Adam Zsolt Wagner, Sergey Shirobokov, Borislav M. Kozlovskii, Francisco J. R. Ruiz, Abbas Mehrabian, M. Pawan Kumar, Abigail See, Swarat Chaudhuri, George Holland, Alex Davies, Sebastian Nowozin, Pushmeet Kohli, Matej Balog, and Google Deepmind. Alphaevolve: A coding agent for ...

    Pith/arXiv arXiv 2025

  21. [22]

    Introducing Operator, January 2025

    OpenAI. Introducing Operator, January 2025. URL https://openai.com/index/ introducing-operator/. Accessed: 2025

    2025

  22. [23]

    Patil, Ion Stoica, and Joseph E

    Charles Packer, Sarah Wooders, Kevin Lin, Vivian Fang, Shishir G. Patil, Ion Stoica, and Joseph E. Gonzalez. Memgpt: Towards llms as operating systems, 2024. URL https: //arxiv.org/abs/2310.08560

    Pith/arXiv arXiv 2024

  23. [24]

    Ui-tars: Pioneering automated gui interaction with native agents.arXiv preprint arXiv:2501.12326, 2025

    Yujia Qin, Yining Ye, Junjie Fang, Haoming Wang, Shihao Liang, Shizuo Tian, Junda Zhang, Jiahao Li, Yunxin Li, Shijue Huang, et al. Ui-tars: Pioneering automated gui interaction with native agents.arXiv preprint arXiv:2501.12326, 2025

    Pith/arXiv arXiv 2025

  24. [25]

    Manning, and Chelsea Finn

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Stefano Ermon, Christopher D. Manning, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model,

  25. [26]

    Proximal policy optimization algorithms

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. InProceedings of the 34th International Conference on Machine Learning (ICML 2017), 2017. URLhttps://arxiv.org/abs/1707.06347

    Pith/arXiv arXiv 2017

  26. [27]

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y . K. Li, Y . Wu, and Daya Guo. Deepseekmath: Pushing the limits of mathematical reasoning in open language models, 2024. URL https://arxiv.org/abs/ 2402.03300

    Pith/arXiv arXiv 2024

  27. [28]

    Reflexion: Language agents with verbal reinforcement learning

    Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. Reflexion: Language agents with verbal reinforcement learning. InProceedings of the 37th Conference on Neural Information Processing Systems (NeurIPS 2023), New Orleans, LA, USA, 2023. URL https://proceedings.neurips.cc/paper_files/paper/2023/ hash/1b44b878bb782e6954cd88862...

    2023

  28. [29]

    David Silver, Aja Huang, Christopher J. Maddison, Arthur Guez, Laurent Sifre, George van den Driessche, Julian Schrittwieser, Ioannis Antonoglou, Veda Panneershelvam, Marc Lanctot, Sander Dieleman, Dominik Grewe, John Nham, Nal Kalchbrenner, Ilya Sutskever, Timothy Lil- licrap, Madeleine Leach, Koray Kavukcuoglu, Thore Graepel, and Demis Hassabis. Masteri...

    2016

  29. [30]

    Promptagent: Strategic planning with language models enables expert-level prompt optimization.arXiv preprint arXiv:2310.16427, 2023

    Xinyuan Wang, Chenxi Li, Zhen Wang, Fan Bai, Haotian Luo, Jiayou Zhang, Nebojsa Jojic, Eric P Xing, and Zhiting Hu. Promptagent: Strategic planning with language models enables expert-level prompt optimization.arXiv preprint arXiv:2310.16427, 2023

    Pith/arXiv arXiv 2023

  30. [31]

    Towards lifespan cognitive systems, 2025

    Yu Wang, Chi Han, Tongtong Wu, Xiaoxin He, Wangchunshu Zhou, Nafis Sadeq, Xiusi Chen, Zexue He, Wei Wang, Gholamreza Haffari, Heng Ji, and Julian McAuley. Towards lifespan cognitive systems, 2025. URLhttps://arxiv.org/abs/2409.13265

    Pith/arXiv arXiv 2025

  31. [32]

    Osworld: Benchmarking multimodal agents for open-ended tasks in real computer environments

    Tianbao Xie, Danyang Zhang, Jixuan Chen, Xiaochuan Li, Siheng Zhao, Ruisheng Cao, Toh Jing Hua, Zhoujun Cheng, Dongchan Shin, Fangyu Lei, Yitao Liu, Yiheng Xu, Shuyan Zhou, Silvio Savarese, Caiming Xiong, Victor Zhong, and Tao Yu. Osworld: Benchmarking multimodal agents for open-ended tasks in real computer environments. In A. Globerson, L. Mackey, D. Bel...

    2024

  32. [33]

    Rag in the wild: On the (in)effectiveness of llms with mixture-of-knowledge retrieval augmentation.arXiv preprint arXiv:2507.20059, 2025

    Ran Xu, Yuchen Zhuang, Yue Yu, Haoyu Wang, Wenqi Shi, and Carl Yang. Rag in the wild: On the (in)effectiveness of llms with mixture-of-knowledge retrieval augmentation.arXiv preprint arXiv:2507.20059, 2025

    Pith/arXiv arXiv 2025

  33. [35]

    A-mem: Agentic memory for llm agents, 2025

    Wujiang Xu, Kai Mei, Hang Gao, Juntao Tan, Zujie Liang, and Yongfeng Zhang. A-mem: Agentic memory for llm agents, 2025. URLhttps://arxiv.org/abs/2502.12110

    Pith/arXiv arXiv 2025

  34. [36]

    Auto-gpt for online decision making: Benchmarks and additional opinions.arXiv preprint arXiv:2306.02224, 2023

    Hui Yang, Sifu Yue, and Yunzhong He. Auto-gpt for online decision making: Benchmarks and additional opinions.arXiv preprint arXiv:2306.02224, 2023. doi: 10.48550/arXiv.2306.02224. URLhttps://doi.org/10.48550/arXiv.2306.02224

    Pith/arXiv arXiv doi:10.48550/arxiv.2306.02224 2023

  35. [37]

    Swe-agent: Agent-computer interfaces enable automated software engineering.Advances in Neural Information Processing Systems, 37:50528–50652, 2024

    John Yang, Carlos E Jimenez, Alexander Wettig, Kilian Lieret, Shunyu Yao, Karthik Narasimhan, and Ofir Press. Swe-agent: Agent-computer interfaces enable automated software engineering.Advances in Neural Information Processing Systems, 37:50528–50652, 2024

    2024

  36. [38]

    React: Synergizing reasoning and acting in language models

    Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models. InProceedings of the 11th International Conference on Learning Representations (ICLR 2023), 2023. URL https: //openreview.net/forum?id=WE_vluYUL-X

    2023

  37. [39]

    τ-bench: A bench- mark for tool-agent-user interaction in real-world domains

    Shunyu Yao, Noah Shinn, Pedram Razavi, and Karthik Narasimhan. τ-bench: A bench- mark for tool-agent-user interaction in real-world domains. InNeurIPS (Workshops), 2024. State-of-the-art agents (e.g. GPT-4o) succeed on <50

    2024

  38. [40]

    typically,

    Yuyan Zhou, Liang Song, Bingning Wang, and Weipeng Chen. Metagpt: Merging large language models using model exclusive task arithmetic. InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1711–1724, Miami, Florida, USA, 2024. Association for Computational Linguistics. doi: 10.18653/v1/2024. emnlp-main.102...

    doi:10.18653/v1/2024 2024

  39. [42]

    Name", "Date

    Detecting Header Rows Overview: To accurately identify header rows, scan the initial region of your dataset. This process is crucial for mapping column information for further processing. Approaches: - Heuristic Checks: - Look for rows where all cells are strings (e.g., "Name", "Date", "Region", " Amount"). - Identify rows with distinctive formatting such...

    2024

  40. [43]

    Name", "Date

    Assigning and Validating Headers Overview: Once headers are detected, assign them programmatically and ensure they match expected schema and data types. Implementation: - Column Naming: - Set names in code, e.g., df.columns = ["Name", "Date", "Region", "Amount"]. - Schema Mapping: - Map headers to a standardized schema, using external files or user prompt...

  41. [44]

    Features: - Freeze Panes: - Automatically freeze header rows in Excel for easier navigation

    Automation and Usability Enhancements Overview: Enhance usability and automation to streamline header extraction and user interaction. Features: - Freeze Panes: - Automatically freeze header rows in Excel for easier navigation. - Highlighting: - Use colored formatting to visually distinguish headers. - Example: - Yellow fill for header row. - Documentatio...

  42. [45]

    Total" or

    Identifying Block Boundaries Overview: Block detection segments data into logical units or tables. Methods: - Boundary Detection: - Find empty rows, repeated labels, or formatting changes. - Example: | Name | Amount | |------|--------| | John | 100 | | | | <-- Empty row indicates new block | Name | Amount | | Alice| 200 | - Machine Learning: - Train class...

  43. [46]

    Actions: - Block ID: - Assign unique IDs (e.g., Block_001, Block_002)

    Processing and Tracking Blocks Overview: Once blocks are detected, assign IDs and enable block-level analysis. Actions: - Block ID: - Assign unique IDs (e.g., Block_001, Block_002). - Analysis: - Perform group-by or aggregation within each block. - Example: - Sum "Amount" for Block_001: 100 + 150 = 250

  44. [47]

    block1.csv

    Additional Block Actions Overview: Enable modular analysis and reporting at the block level. Features: - Summary Rows: - Add computed totals/averages for each block. - Export/Save: - Save blocks as separate files or sheets. - Example: - Export Block_001 to "block1.csv" Search for Values or Patterns

  45. [48]

    North" in the

    Search Execution Methods Overview: Efficiently locate specific values or patterns in your data. Techniques: - Manual Tools: - Use Ctrl + F in Excel for quick lookups. - Programmatic Search: - Scan all cells using loops or vectorized code. - Example: - Find all instances of "North" in the "Region" column. - Pattern Matching: - Support exact, wildcard (*Tot...

  46. [49]

    26 Actions: - Logging: - Record coordinates (e.g., Sheet1, Row 3, Col "Region")

    Recording and Highlighting Results Overview: Log and visualize search matches for user review. 26 Actions: - Logging: - Record coordinates (e.g., Sheet1, Row 3, Col "Region"). - Highlighting: - Apply conditional formatting to search hits

  47. [50]

    John" to

    Advanced Search Scenarios Overview: Handle complex or large-scale search requirements. Scenarios: - Merged Cells: - Search within merged cells or across multiple sheets. - Export: - Export found results for further analysis. - Example: - Export all rows containing "John" to "john_results.csv" Writeback Results

  48. [51]

    Approved_Flag

    Output Placement Overview: Choose where and how to insert results. Options: - Target Columns: - Select existing or blank columns for output. - Appending: - Add new columns for flags, counts, or statuses. - Example: - Add "Approved_Flag" column next to "Status"

  49. [52]

    Success", red for

    Writing and Styling Results Overview: Automate and style the output for visibility. Methods: - Formulas/Code: - Use code (e.g., ws.cell(row, col).value = result) to insert results. - Styling: - Bold, borders, or colors for output cells. - Example: - Green fill for "Success", red for "Error"

  50. [53]

    2024-06-01, User: admin

    Audit and Protection Overview: Maintain the integrity and traceability of results. Measures: - Lock Columns: - Prevent edits to output columns. - Timestamps/User Info: - Add audit trail for writebacks. - Example: - "2024-06-01, User: admin" Difference in State

    2024

  51. [54]

    Process: - Load Sheets: - Read both sheets into memory

    Sheet Comparison Overview: Identify changes between input and output sheets. Process: - Load Sheets: - Read both sheets into memory. - Compare Cells: - Detect differences by position and value. 27

  52. [55]

    North"→"South

    Recording and Reporting Differences Overview: Log and report all detected changes. Actions: - Log Mismatches: - Record cell coordinates and values. - Example: - Cell B3: "North"→"South" - Export Diff Report: - List all detected differences for review

  53. [56]

    Features: - Highlight Changes: - Color code changed cells

    Visualization and Automation Overview: Make changes visible and automate validation. Features: - Highlight Changes: - Color code changed cells. - Automate Checks: - Integrate diff comparisons into test scripts. Column Selection

  54. [57]

    ID", "Date

    Selection Criteria Overview: Choose relevant columns for analysis. Methods: - Labels/Indices: - Select by name or position. - Dynamic Rules: - E.g., all numeric columns. - Assign Roles: - Example: "ID", "Date", "Metric"

  55. [58]

    Actions: - Rename/Relabel: - Standardize column names

    Preparation and Validation Overview: Prepare columns for consistent use. Actions: - Rename/Relabel: - Standardize column names. - Validate Types: - Ensure columns are of expected type. - Example: - "Date" column as datetime

  56. [59]

    Features: - Presets: - Save selection profiles

    Reusability Overview: Save and reuse column selections. Features: - Presets: - Save selection profiles. - Downstream Use: - Use validated columns in subsequent processes. Filter Rows

  57. [60]

    Approved

    Filtering Methods Overview: Refine your dataset with filters. Techniques: - Spreadsheet Tools: 28 - Use built-in filters. - Code Logic: - Filter with code (e.g., df[df[’Status’] == ’Approved’]). - Multiple Criteria: - Combine conditions (AND/OR). - Example: - Status = "Approved" AND Amount > 100

  58. [61]

    Actions: - Helper Columns: - Compute intermediate flags

    Helper Columns and Complex Filters Overview: Simplify filtering using helper columns. Actions: - Helper Columns: - Compute intermediate flags. - Document Logic: - Record filtering rules for audit

  59. [62]

    Features: - Highlighting: - Grey-out filtered-out rows

    Post-Filter Actions Overview: Visualize and export filtered data. Features: - Highlighting: - Grey-out filtered-out rows. - Export: - Save the filtered dataset. Merge Tables

  60. [63]

    Customer_ID

    Key-Based Merging Overview: Combine tables using shared keys. Techniques: - Join Operations: - Use VLOOKUP, JOIN, or code merges. - Example: - Merge "Customer_ID" from two tables. - Align Data: - Match on columns like "ID", "Name"

  61. [64]

    Methods: - Vertical Append: - Combine rows from similar tables

    Stack-Based Merging Overview: Append tables when keys ’arent needed. Methods: - Vertical Append: - Combine rows from similar tables. - Deduplicate: - Remove duplicate records

  62. [65]

    Actions: - Source Column: - Add "Source" to indicate origin

    Tracking and Audit Overview: Track source and unmatched records. Actions: - Source Column: - Add "Source" to indicate origin. - Highlight Unmatched: - Mark or export mismatched rows. Pivot or Unpivot

  63. [66]

    Amount" by

    Pivoting Data Overview: Summarize data using pivots. 29 Methods: - PivotTables: - Group by row/column dimensions. - Example: - Sum "Amount" by "Region". - Aggregation: - Choose SUM, AVG, COUNT, etc

  64. [67]

    Techniques: - Melt Operations: - Convert columns into rows

    Unpivoting (Melting) Data Overview: Reshape data from wide to long format. Techniques: - Melt Operations: - Convert columns into rows. - Example: - | Year | Sales_2019 | Sales_2020 | |------|------------|------------| → | Year | Sales_Year | Value | - Flexible Restructuring: - Selectively unpivot non-ID columns

  65. [68]

    Features: - Flatten Pivot Table: - Convert back to flat for further analysis

    Post-Pivot Actions Overview: Prepare pivoted data for export. Features: - Flatten Pivot Table: - Convert back to flat for further analysis. - Reorder/Rename: - Clarify pivoted fields. Map with Lookup Tables

  66. [69]

    N"→Label

    Mapping Techniques Overview: Standardize data using lookups. Methods: - Functions: - Use VLOOKUP, merge with dictionaries. - Code-to-Label: - Example: - Code "N"→Label "North"

  67. [70]

    Actions: - Apply Mappings: - Across selected columns

    Application and Fallbacks Overview: Apply lookups and handle missing values. Actions: - Apply Mappings: - Across selected columns. - Handle Missings: - Use defaults for missing codes

  68. [71]

    Features: - Cache Mappings: - Store for repeated use

    Audit and Display Overview: Ensure mapping transparency. Features: - Cache Mappings: - Store for repeated use. - Display Codes/Labels: - Show both for clarity. 30 Fill Missing Data

  69. [72]

    Techniques: - Forward/Backward Fill: - Fill gaps with prior/next value

    Choosing Fill Methods Overview: Impute missing data appropriately. Techniques: - Forward/Backward Fill: - Fill gaps with prior/next value. - Default Values: - Use fixed placeholder (e.g., 0, "Unknown"). - Contextual Example: - Dates: Fill missing month with last known month

  70. [73]

    Actions: - Targeted Filling: - Apply to specific columns/rows

    Application and Auditing Overview: Apply fills and flag for review. Actions: - Targeted Filling: - Apply to specific columns/rows. - Flag Filled Cells: - Highlight for later review

  71. [74]

    Features: - Record Logic: - Document assumptions and methods

    Documentation Overview: Keep fill logic transparent. Features: - Record Logic: - Document assumptions and methods. - Audit Trail: - Track all changes. Flag Rows or Cells

  72. [75]

    Examples: - Simple Rule: - Flag where Amount < 0 - Complex Rule: - Flag where Status = "Pending" and Amount > 1000

    Defining Flag Rules Overview: Establish criteria for flagging. Examples: - Simple Rule: - Flag where Amount < 0 - Complex Rule: - Flag where Status = "Pending" and Amount > 1000

  73. [76]

    Flag" column with

    Applying Flags Overview: Insert flags and summarize. Actions: - Flag Column: - Add "Flag" column with "Yes"/"No". - Export Flagged Rows: - Save for further inspection

  74. [77]

    Features: - Multi-Criteria: - Combine several rules for granular checks

    Advanced Flagging Overview: Use multiple criteria and document. Features: - Multi-Criteria: - Combine several rules for granular checks. - Notes: - Document flagging rationale. 31 Sort Data

  75. [78]

    Region", then by

    Setting Sort Criteria Overview: Organize data for analysis. Options: - Sort Columns: - By value, ascending/descending. - Multi-Level: - E.g., sort by "Region", then by "Amount"

  76. [79]

    Methods: - Spreadsheet Tools: - Built-in sort features

    Applying Sorts Overview: Implement sorting programmatically or manually. Methods: - Spreadsheet Tools: - Built-in sort features. - Code: - E.g., df.sort_values([’Region’, ’Amount’])

  77. [80]

    Actions: - Renumber Rows: - Update indices

    Post-Sort Actions Overview: Finalize sorted data. Actions: - Renumber Rows: - Update indices. - Highlight Extremes: - Mark top/bottom values. Validate Data

  78. [81]

    Amount" > 0. - Pattern: - Date columns match YYYY-MM-DD. - Business Rule Example: -

    Validation Checks Overview: Ensure data meets required standards. Checks: - Type: - Ensure numeric columns contain numbers. - Range: - E.g., "Amount" > 0. - Pattern: - Date columns match YYYY-MM-DD. - Business Rule Example: - "Start Date" < "End Date"

  79. [82]

    Actions: - Highlight Invalids: - Color-code errors

    Marking and Reporting Overview: Visualize and report errors. Actions: - Highlight Invalids: - Color-code errors. - Export Summary: - Table of error counts and locations

  80. [83]

    Features: - Pre-Processing Step: - Validate before analysis

    Integration in Workflow Overview: Make validation a routine part of processing. Features: - Pre-Processing Step: - Validate before analysis. - Automation: - Integrate into data pipelines. 32 Split Sheets or Data

Showing first 80 references.