{"id":"ad53a4fb-8596-404c-b187-5e442fc56cd9","arxiv_id":"2610.23098","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"InstAgent organizes heterogeneous robot teams as behavior trees with recursive self-improvement, preserving execution feedback as reusable programs and achieving consistent task success gains across three LLM backbones.","lead":"InstAgent is an agent operating system that uses behavior trees to coordinate heterogeneous robots, with a recursive self-improvement mechanism that retains execution corrections as reusable experience. In 20 real-world multi-robot tasks, it improved task success by 15 to 25 percentage points across three different language models compared to starting from scratch each time.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek/deepseek-v4-flash","headline":"The paper does not describe how the historical feedback and cases used to build the E1 and E2 libraries were generated, leaving it unclear whether the E1 vs E2 comparison reflects automated RSI or human-crafted revisions.","rationale":"The reader identified the representativeness of the 20 tasks as the weakest assumption. That is a fair concern, but the paper itself acknowledges this limitation (sentence b230). I identify a more specific and potentially more serious concern: the lack of transparency in the construction of the experience library, which is the direct evidence for the benefit of retained revisions. Without knowing whether the historical feedback and validated revisions were produced autonomously by the system, the central claim that InstAgent 'recursively self-improves' is not empirically established. The paper should disclose this information. This does not invalidate the work but increases the condition of acceptance. Therefore, I recommend keeping the CONDITIONAL verdict, as the paper is otherwise sound and the results are consistent, but this missing detail must be clarified to fully evaluate the claim.","tokens_in":15045,"tokens_out":4483,"duration_ms":59679,"concrete_test":"Request a detailed description of the experience library construction process: (a) Source and number of prior runs used to generate historical feedback; (b) Whether those runs were executed autonomously by the system or directed by humans; (c) How 'case availability' was determined; (d) Whether the validated revisions in the E2 library were output by Algorithm 1 or hand-curated. If the library was built autonomously from runs on tasks not overlapping with the 20 evaluation tasks, the claim is strengthened. If human intervention was involved, the E1–E2 comparison no longer cleanly measures automated RSI.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim that retained and updated procedural experience improves task success rests on the comparison between Fixed-Experience BT (E1) and RSI-Updated Experience BT (E2). The paper states that 'E1 and E2 share historical feedback and cases; only E2 persists validated revisions to inherited subtrees before evaluation' and that 'evaluation tasks are excluded from library construction.' However, the source and construction method of the historical feedback and cases are never described. If these were generated by the system in prior runs on similar tasks via autonomous execution and revision (Algorithm 1), the E1–E2 comparison cleanly demonstrates the benefit of retaining revisions. If instead the historical feedback was hand-crafted or derived from manual analysis of known failure modes, then the observed gains conflate system self-improvement with human engineering. The paper provides no description of the number, nature, or autonomy of the runs that produced the library, nor does it specify whether the validated revisions in E2 were produced by the system or by the experimenters. This gap undermines the support for the claim of automated recursive self-improvement.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek/deepseek-v4-flash","summary":"This paper presents InstAgent, an Agent Operating System for heterogeneous multi-robot collaboration. The system organizes robot behaviors as agentic behavior trees (BTs) that encode inter-robot dependencies, handoff conditions, and recovery logic. It includes a recursively self-improving (RSI) mechanism that revises these BTs from execution feedback and stores validated revisions in an experience library for reuse across tasks. A shared multimodal spatio-temporal memory (SMAR-T) provides contextual observations for planning and failure diagnosis. Experiments on 20 real-world multi-robot tasks with three language models (GPT-5.6, DeepSeek-V4, Qwen3.8) show that the full system (E2) achieves higher task success rates (e.g., 38/40 for GPT-5.6) compared to baselines without cross-task experience (BT Recovery E0, e.g., 32/40), with improvements of 15–25 percentage points. Additional RAVEN-QA experiments on memory retrieval report 93.14% overall accuracy.","tokens_in":15428,"tokens_out":2354,"duration_ms":30314,"significance":"If the results hold, InstAgent advances the state of the art in heterogeneous multi-robot collaboration by providing a structured program-level mechanism for retaining and reusing corrected coordination procedures across tasks. The combination of behavior-tree representation, recursive self-improvement, and shared spatio-temporal memory is a novel synthesis that addresses a known gap in prior work (e.g., COHERENT, LLM-HBT, ABot-AgentOS). The systematic evaluation across three models with paired trials and statistical tests is a strength. The RAVEN-QA results also demonstrate a competitive memory retrieval approach. The work has practical relevance for long-horizon, multi-robot service tasks.","major_comments":[{"comment":"The central E1–E2 comparison is intended to isolate the benefit of retaining validated revisions, but the paper does not describe how the historical feedback and cases used to build the E1 and E2 libraries were generated. The text states (caption of Table II) that 'E1 and E2 share historical feedback and cases; only E2 persists validated revisions to inherited subtrees before evaluation.' However, whether these were generated autonomously by the system on prior runs (through Algorithm 1) or manually crafted/hand-selected is never explained. If they were hand-crafted, then the E1–E2 comparison conflates system self-improvement with human engineering, undermining the claim of automated recursive self-improvement. The authors must specify the origin, number, and generation process of these historical entries, and discuss how this affects the interpretation of the RSI claim.","section":"§IV-A, Table II, and §III-C","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The experimental evaluation is limited to 20 tasks each run twice (40 runs per model-configuration) on one fixed robot team (G1, Go2, ALOHA). While the paper acknowledges this limitation (sentence b230), the claim of 'consistent gains' depends on the tasks and perturbations being representative. The injection of perturbations (excessive arrival error, navigation/loading failures, destination changes) is described, but the number of distinct task templates and the variability in subtrees inherited are not reported. The authors should provide a description of task diversity (e.g., how many distinct BT structures appear, how many subtrees are shared across tasks) to help assess generalizability.","section":"§IV-A, Table II","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The semantic retrieval lambda parameter (λ=0.15) and the budget/candidate limits (recovery budget=5, candidate limit J_max=3) are fixed values with no sensitivity analysis or justification. Since the retrieval quality affects both BT construction and revision, a sensitivity study over λ (e.g., 0.0, 0.15, 0.5, 1.0) and at least one budget variation is needed to demonstrate that the reported gains are not brittle to these choices.","section":"§IV-A, Eq. (4) and free parameters","authors_acknowledge":null,"authors_acknowledge_p":null}],"minor_comments":[{"comment":"The 'safe stopping is unconfirmed' condition on line 8 is left undefined. What constitutes confirmed safe stopping? Please clarify the criterion and how it differs from the recovery budget exhaustion.","section":"§III-C, Algorithm 1","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The description of candidate revisions (three per failure episode, counter reset per episode) should be stated more clearly; it is currently split across two paragraphs and could be consolidated for readability.","section":"§IV-A, 'Success criteria and recovery budgets'","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The illustrative case describes a revision in which VLM checks were added. It would be helpful to report whether this revision was generated autonomously by the system (via ReviseAndCheck in Algorithm 1) or written by the experimenters. The text says 'Reflection and replanning added VLM checks,' but it is ambiguous whether this was performed by the MLLM within the RSI loop.","section":"§IV-B, 'Collaboration-program revision and reuse'","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The font size in the figures is very small; the labels (e.g., 'Skill BTs', 'HeteroComposer', 'RSI Engine') are hard to read in print. Consider enlarging or providing vector versions.","section":"Figures 1 and 2","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The semantic tool description uses tilde notation for normalization but does not explicitly state that normalization is over the C10 set only. This is clear from context but could be stated for precision.","section":"§III-D, 'Semantic, spatial, and temporal retrieval tools'","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"Several cited works (e.g., [25]) are listed as arXiv preprints with potential subsequent publication dates. Please update to published versions where available, and ensure all citations follow the journal style.","section":"References","authors_acknowledge":null,"authors_acknowledge_p":null}],"recommendation":"major_revision"},"author_rebuttal":null,"desk_editor":{"model":"deepseek/deepseek-v4-flash","letter":"Here's my read on the InstAgent paper.\n\nThe paper builds a genuine integrated system for heterogeneous robot teams that combines behavior trees, recursive self-improvement (RSI), and shared spatio-temporal memory. What's new is the specific way they preserve validated collaboration subtrees as reusable BT structures that later tasks can inherit and adapt. The closed-loop composition–execution–revision–retention cycle is cleanly described and convincingly implemented. The system addresses real limitations in prior work: corrections are typically not retained as reusable programs, inter-robot dependencies are underspecified, and contextual memory for handoffs is weak.\n\nThe real-robot evaluation is the paper's strongest feature. They run 20 tasks with three LLMs, compare four experience configurations, and report paired statistical tests. The progression from no cross-task experience (E0) to fixed experience (E1) to self-updated experience (E2) shows monotonic improvement in success rate and first-attempt success across all models. The gains are substantial — 15 to 25 percentage points — and the McNemar and Wilcoxon p-values are below 0.05. The RAVEN-QA memory evaluation is a nice addition, showing that their agent-driven retrieval pipeline outperforms both RA VEN and ReMEmbR on the same protocol.\n\nSoft spots. First, the evaluation scope is narrow: 20 tasks on a single robot team, with perturbations selected by the experimenters. The paper acknowledges this in the future work sentence, but it means the central claim of consistent gains is not yet generically established. Second, the provenance of the initial experience library (the \"historical feedback and cases\" shared by E1 and E2) is not described. The paper says both libraries are frozen before evaluation and that evaluation tasks are excluded from construction, but it doesn't say whether the library was built from prior autonomous runs, from hand-crafted examples, or from some other process. This is a gap, though not a fatal one: the primary comparison (E0 vs E2) shows that the full system works even if the initial library was hand-crafted, and the E1–E2 comparison still demonstrates that the retention of automated revisions helps beyond a fixed baseline. Still, the paper would be stronger if it explained how the seed library was produced. Third, the RSI mechanism uses the same LLM for planning and revision, which creates a risk of self-reinforcement; the paper partially mitigates this by comparing against fixed-experience baselines using the same LLM, but I'd like to see a variant that seeds revisions with a different model or with random perturbations.\n\nOverall, this is a solid systems paper with a useful architecture and careful real-world experiments. I'd send it to peer review. It's not groundbreaking but it is a well-integrated contribution that pushes on a real problem. The main limitation — narrow evaluation — is openly stated and is typical for real-robot work. I'd recommend conditional acceptance with a request for clarification on the library construction and ideally a public release of code and data.","headline":"Solid system paper on heterogeneous robot collaboration with recursive self-improvement — evaluation is careful but scope is limited.","tokens_in":15913,"tokens_out":3017,"duration_ms":33870},"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek/deepseek-v4-flash","headline":"InstAgent, a recursively self-improving Agent Operating System, enables heterogeneous robot teams to retain and reuse execution-time corrections as behavior trees, improving task success by 15–25 percentage points across three language…","keywords":["heterogeneous robot collaboration","recursive self-improvement","behavior trees","agent operating system","multi-robot handoff","spatio-temporal memory","experience reuse","LLM-based robotics"],"falsifier":"If a broader set of tasks (e.g., 100+ tasks with randomized compositions, failure types, and robot team configurations) showed no statistically significant improvement in task success between Full InstAgent and BT Recovery (E0), then the central claim that retained and updated procedural experience provides consistent gains would be refuted.","tokens_in":16571,"feed_emoji":"🤖","tokens_out":3274,"duration_ms":43696,"temperature":0.7,"pith_summary":"Heterogeneous robot teams performing long-horizon tasks often fail on initial attempts, and existing systems lack a mechanism to turn execution-time corrections into reusable experience. InstAgent addresses this by organizing team behaviors as agentic behavior trees (BTs) that explicitly encode capability assignments, inter-robot dependencies, and handoff checks. Its recursive self-improvement (RSI) mechanism revises these collaboration programs from execution feedback, validates the revisions, and stores them as executable subtrees with linked documents for future reuse. Shared multimodal spatio-temporal memory (SMAR-T) provides task-relevant context across space and time to support planning and handoff verification. In experiments on 20 real-world multi-robot tasks with three high-level models, InstAgent consistently outperformed baseline approaches, achieving higher task success and first-attempt success while reducing the number of failure-triggered recoveries.","feed_headline":"Self-improving robot teams boost success 25%","feed_subtitle":"InstAgent stores and reuses execution fixes as behavior trees, cutting recoveries by over 60% in 20 real-world tasks.","key_machinery":"The key machinery is the combination of an agentic behavior tree (BT) representation for heterogeneous collaboration and the Collab-RSI (recursive self-improvement) mechanism. Behavior trees provide a structured, modular control flow that explicitly encodes capability assignment, inter-robot dependencies, handoff conditions, and recovery logic. Collab-RSI uses execution feedback to identify and revise failed subtrees, validates the changes through re-execution, and retains validated BT subtrees together with context documents in an experience library. This allows later tasks to inherit and adapt these subtrees, forming a closed loop of composition, execution, verification, revision, and reuse. Shared Multimodal Spatio-Temporal Memory with Agent-Driven Retrieval (SMAR-T) supports both initial planning and failure diagnosis by retrieving historical observations according to semantic, spatial, and temporal constraints.","core_discovery":"The central claim is that collaboration programs for heterogeneous robot teams can be expressed as behavior trees that are recursively improved through execution feedback, with validated revisions persistently retained as reusable experience for subsequent tasks. InstAgent demonstrates this by composing atomic skill BTs into a collaborative BT that includes handoff conditions and recovery branches. When execution fails, the Collab-RSI mechanism diagnoses the failure, revises the relevant subtree (e.g., adding a verification check or adjusting a dependency), validates the revision by re-execution, and writes the updated BT and its paired document back into the experience library. This closes the loop: later tasks inherit and adapt previously validated subtrees, accumulating executable collaboration experience across the team. The paper shows that over 20 tasks, the full InstAgent system raises task success by 15–25 percentage points compared to BT execution without cross-task procedural experience, and also improves first-attempt success and reduces recovery counts across three different language models.","pith_inferences":["A natural extension is to test whether the same RSI mechanism works when the robot team composition changes (e.g., adding or removing a robot type), which would indicate whether the experience library generalizes beyond fixed teams.","The paper's limitation of 20 hand-designed tasks suggests that a more diverse benchmark (e.g., randomized task compositions and failure modes) is needed to estimate the true coverage of the claimed improvements; if the gains shrink on out-of-distribution tasks, the method may rely heavily on the specific perturbations tested.","Longer deployment studies could reveal whether retained experience degrades over time due to changing environments or sensor drift, or whether the recursive update process leads to over-specialized subtrees that hurt performance on novel variations."],"forward_implications":["If InstAgent is correct, heterogeneous robot teams can autonomously accumulate procedural experience from execution, reducing the need for manual reprogramming when new tasks share structural similarities with previously solved ones.","The explicit retention of validated handoff checks and recovery logic as reusable subtrees means that improvements made during one task directly transfer to future tasks, enabling continual improvement in team coordination.","The SMAR-T memory system, when combined with the Agent-driven retrieval loop, provides a method for robots to ground planning and handoff verification in past observations across space and time, which could improve reliability in long-horizon tasks.","The demonstrated gains across three language models (GPT-5.6 Sol, DeepSeek-V4-Pro, Qwen3.8-27B) suggest that the approach is model-agnostic and can leverage improvements in underlying language models without changing the system architecture."],"supporting_citations":[{"why":"Provides the foundational definition of behavior trees in robotics as the control structure for reactivity and modularity.","marker":"[19]"},{"why":"Introduces LLM-HBT, a prior method that uses LLMs to construct dynamic behavior trees for heterogeneous robot coordination, which serves as a baseline and context for InstAgent's contributions.","marker":"[10]"},{"why":"Defines Experience-Context RSI through Reflexion, storing feedback as verbal reflections, which InstAgent extends to heterogeneous collaboration programs.","marker":"[1]"},{"why":"Introduces Voyager for structured-program RSI in single-agent settings, which InstAgent adapts to multi-robot collaboration BTs.","marker":"[2]"},{"why":"Describes COHERENT, a centralized task assigner for heterogeneous robots, which InstAgent contrasts by explicitly representing inter-robot dependencies and handoffs in the BT structure.","marker":"[9]"},{"why":"Presents RA VEN, a visuo-spatio-temporal memory system for robot navigation, which InstAgent compares against in the RA VEN-QA retrieval evaluation.","marker":"[6]"}],"fun_headline_variants":["InstAgent boosts multi-robot success by 15-25%","Self-improving behavior trees cut robot recoveries 60%","InstAgent stores reusable execution fixes as behavior trees","Recursively improving OS boosts heterogeneous robot collaboration","Agent OS with memory improves multi-robot task success"],"cache_read_input_tokens":0,"weakest_assumption_plain":"The experimental evaluation assumes that the 20 hand-designed tasks and the injected perturbations (excessive arrival error, navigation and loading failures, user-requested destination changes) adequately represent the range of challenges encountered in real-world heterogeneous multi-robot collaboration, and the paper explicitly acknowledges that future work should examine transfer to new task compositions and robot teams.","fun_headline_variants_meta":{"raw":{"variants":["InstAgent boosts multi-robot success by 15-25%","Self-improving behavior trees cut robot recoveries 60%","InstAgent stores reusable execution fixes as behavior trees","Recursively improving OS boosts heterogeneous robot collaboration","Agent OS with memory improves multi-robot task success"]},"model":"deepseek/deepseek-v4-flash","effort":"low","cost_usd":8.3e-05,"raw_usage":{"cost":8.33868e-05,"is_byok":false,"cost_details":{"upstream_inference_cost":8.33868e-05,"upstream_inference_prompt_cost":1.36836e-05,"upstream_inference_completions_cost":6.97032e-05},"total_tokens":3210,"prompt_tokens":905,"completion_tokens":2305,"prompt_tokens_details":{"audio_tokens":0,"video_tokens":0,"cached_tokens":0,"cache_write_tokens":0},"completion_tokens_details":{"audio_tokens":0,"image_tokens":0,"reasoning_tokens":2225}},"tokens_in":905,"tokens_out":2305,"cost_source":"openrouter_usage","duration_ms":28055,"image_pages":[],"temperature":1.0,"billed_attempts":1,"reasoning_tokens":2225,"transport_provider":"openrouter","cache_read_input_tokens":0,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-10-19T15:53:06.489156+00:00","model_set":{"reader":"deepseek/deepseek-v4-flash"},"pipeline_version":"pith-pipeline@v1.4.1-daily-deepseek-or-medium","input_kind":null,"input_chars":null,"input_total_chars":null,"passes_missing":null,"published_standing":null,"published_confidence":null,"objection_audit":null,"run_record":null,"basis":{"kind":"not_recorded","source":null,"chars_given":null,"chars_total":null},"receipt":null,"falsifier":"If a broader set of tasks (e.g., 100+ tasks with randomized compositions, failure types, and robot team configurations) showed no statistically significant improvement in task success between Full InstAgent and BT Recovery (E0), then the central claim that retained and updated procedural experience provides consistent gains would be refuted.","supporting_citations":[{"cited_title":"Towards blended reac- tive planning and acting using behavior trees,","cited_arxiv_id":null,"evidence_quote":"Provides the foundational definition of behavior trees in robotics as the control structure for reactivity and modularity."},{"cited_title":"Llm-hbt: Dynamic behavior tree construction for adaptive coordination in heterogeneous robots,","cited_arxiv_id":null,"evidence_quote":"Introduces LLM-HBT, a prior method that uses LLMs to construct dynamic behavior trees for heterogeneous robot coordination, which serves as a baseline and context for InstAgent's contributions."},{"cited_title":"Reflexion: Language agents with verbal reinforcement learning,","cited_arxiv_id":null,"evidence_quote":"Defines Experience-Context RSI through Reflexion, storing feedback as verbal reflections, which InstAgent extends to heterogeneous collaboration programs."},{"cited_title":"Voyager: An Open-Ended Embodied Agent with Large Language Models","cited_arxiv_id":"2305.16291","evidence_quote":"Introduces Voyager for structured-program RSI in single-agent settings, which InstAgent adapts to multi-robot collaboration BTs."},{"cited_title":"Coherent: Collaboration of heterogeneous multi-robot system with large language models,","cited_arxiv_id":null,"evidence_quote":"Describes COHERENT, a centralized task assigner for heterogeneous robots, which InstAgent contrasts by explicitly representing inter-robot dependencies and handoffs in the BT structure."},{"cited_title":"RAVEN: Long-Horizon Reasoning & Navigation with a Visuo-Spatio-Temporal Memory","cited_arxiv_id":"2606.25206","evidence_quote":"Presents RA VEN, a visuo-spatio-temporal memory system for robot navigation, which InstAgent compares against in the RA VEN-QA retrieval evaluation."}],"lane_accuracy":{"status":"not yet measured","lane":{"model":"deepseek/deepseek-v4-flash","input_kind":null},"url":"https://pith.science/accuracy"},"review_version":1}