{"id":"66836ba9-b127-4305-b861-4155068e3921","arxiv_id":"2606.19911","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":5.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":0,"one_line_summary":"MATM is a retrieval framework that lets populations of LLM agents share and reuse task trajectories to improve performance on interactive tasks without joint training.","lead":"The paper proposes Multi-Agent Transactive Memory (MATM), a shared repository where LLM agents store and retrieve task trajectories to help other agents solve similar problems in environments like ALFWorld and WebArena. A smart generalist might read it to understand how agent populations could accumulate and reuse procedural knowledge without retraining or explicit coordination.","discovery_kind":"new_application","skeptic_critique":{"model":"grok-4.3","headline":"No significant objection identified","rationale":"Reader correctly flagged abstract-only review as limiting verification. With full text now referenced, the argument structure contains no load-bearing gap that would alter the UNVERDICTED status; the empirical claim can be checked directly via reproduction rather than by attacking an unstated assumption.","tokens_in":1659,"tokens_out":239,"duration_ms":13516,"concrete_test":"Reproduce the reported ALFWorld and WebArena results using the exact retrieval index and consumer-agent prompts described in the full manuscript; confirm that performance deltas and step reductions hold under the same evaluation protocol.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim rests on empirical demonstration that retrieved trajectories improve performance in ALFWorld and WebArena without coordination or joint training. The provided abstract and strongest claim are consistent with a retrieval-augmented design pattern; the weakest_assumption (reusable procedural knowledge transferable across agents) is a standard premise for such systems and does not introduce an internal inconsistency or untestable circularity. No parameter-free derivation or formal claim is present that would require additional scrutiny beyond the reported experiments.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The manuscript proposes Multi-Agent Transactive Memory (MATM), a framework extending retrieval-augmented generation to agent-generated trajectories. Producer agents store trajectories encoding procedural knowledge in a shared repository; consumer agents retrieve them to improve execution in interactive environments (ALFWorld, WebArena). The central empirical claim is that retrieval improves downstream task performance and reduces interaction steps without coordination or joint training across heterogeneous agents.","tokens_in":1761,"tokens_out":412,"duration_ms":14169,"significance":"If the empirical results hold with proper controls, MATM offers a practical, coordination-free design pattern for experience reuse across agent populations and model families. This extends RAG concepts to multi-agent settings and could support scalable open ecosystems where newly instantiated agents avoid rediscovering solutions.","major_comments":[{"comment":"Abstract and Experiments section: the manuscript states that retrieving trajectories improves performance and reduces steps on ALFWorld and WebArena, yet provides no quantitative metrics, error bars, baseline comparisons, retrieval implementation details, or statistical tests. Without these, the magnitude and reliability of the claimed gains cannot be assessed and post-hoc selection of trajectories remains possible.","section":"Abstract, Experiments"},{"comment":"Methods/Implementation: the weakest assumption—that agent trajectories encode reusable procedural knowledge transferable across different agent instances or model families without adaptation or filtering—is not tested via ablation on cross-model transfer or negative examples; this is load-bearing for the population-level claim but lacks supporting controls.","section":"Methods"}],"minor_comments":[{"comment":"Notation for the shared repository and retrieval process is introduced informally; a formal definition or pseudocode would improve clarity.","section":"Framework"},{"comment":"The abstract claims 'without coordination or joint training' but does not explicitly contrast against coordinated or jointly trained baselines in the reported experiments.","section":"Abstract"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for the constructive feedback, which identifies key areas where the empirical claims can be presented more rigorously. We address each major comment below.","responses":[{"response":"We agree that the abstract summarizes results qualitatively and that the Experiments section requires more explicit reporting to substantiate the claims. In the revised manuscript we will update the abstract with specific quantitative metrics (success rates, step reductions), include error bars, baseline comparisons, retrieval implementation details, and statistical tests. This will also clarify how trajectories were selected to mitigate concerns about post-hoc selection.","revision_made":"yes","referee_comment":"[Abstract, Experiments] Abstract and Experiments section: the manuscript states that retrieving trajectories improves performance and reduces steps on ALFWorld and WebArena, yet provides no quantitative metrics, error bars, baseline comparisons, retrieval implementation details, or statistical tests. Without these, the magnitude and reliability of the claimed gains cannot be assessed and post-hoc selection of trajectories remains possible."},{"response":"We acknowledge that the current experiments do not include explicit ablations for cross-model transfer or negative examples, which would strengthen support for the transferability assumption. We will add these controls (e.g., cross-model trajectory reuse and negative-example filtering) to the Methods and Experiments sections in the revision.","revision_made":"yes","referee_comment":"[Methods] Methods/Implementation: the weakest assumption—that agent trajectories encode reusable procedural knowledge transferable across different agent instances or model families without adaptation or filtering—is not tested via ablation on cross-model transfer or negative examples; this is load-bearing for the population-level claim but lacks supporting controls."}],"tokens_in":1289,"tokens_out":358,"duration_ms":20682,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"The core point is that this paper shows how to store agent trajectories in a shared repository so other agents can retrieve them and skip re-solving the same procedural steps. It works without any coordination or joint training, tested in ALFWorld and WebArena.\n\nWhat is new is the population-level framing: instead of each agent discarding its trajectory after one run, the system treats those trajectories as reusable artifacts for heterogeneous agents. The choice of long-horizon interactive environments is sensible because those trajectories carry the richest structure. The design keeps things simple and decentralized, which fits the stated goal of open ecosystems.\n\nThe paper does a solid job naming the practical problem of repeated computation across agent instances and offering a retrieval-based fix that builds directly on existing RAG methods. The claim that this reduces interaction steps is consistent with how retrieval usually helps.\n\nThe soft spots are mostly around evidence. The abstract states performance gains but supplies no quantitative results, baselines, or retrieval details, so it is difficult to judge how large or reliable the improvement is. The key assumption—that trajectories encode transferable procedural knowledge across different model families—makes sense in principle but would benefit from more direct checks on when retrieval actually helps versus when it adds noise. Minor issues include the lack of error bars or ablation on the retrieval implementation itself.\n\nThis is for researchers working on agent infrastructure and multi-agent deployment who care about reducing redundant work. A reader looking for a straightforward design pattern rather than a theoretical advance would find it useful. It deserves a serious referee because the idea is testable on standard benchmarks and the central empirical claim can be checked with the right controls.","headline":"MATM is a clean extension of RAG to agent trajectories that addresses redundant rediscovery in open agent populations, though the size of the gains remains unclear without numbers.","tokens_in":2258,"tokens_out":404,"would_cite":false,"duration_ms":16236,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"A shared repository lets agents retrieve and reuse trajectories produced by other agents to improve task performance and reduce steps.","keywords":["multi-agent systems","trajectory retrieval","procedural knowledge","knowledge sharing","interactive environments","retrieval-augmented agents","transactive memory"],"falsifier":"A controlled test in which agents given retrieved trajectories show equal or lower success rates and equal or more steps than identical agents given no retrieval would falsify the performance claim.","tokens_in":2577,"feed_emoji":"🤖","tokens_out":562,"duration_ms":18266,"temperature":0.7,"pith_summary":"The paper argues that agent trajectories encode reusable procedural knowledge that is typically wasted when each new agent starts fresh. By storing trajectories in a central repository that any agent can query, the system allows consumer agents to borrow solutions from producer agents. Experiments in ALFWorld and WebArena show higher success rates and fewer interaction steps when retrieval is used, even though agents come from separate instances and receive no joint training or coordination. A reader would care because the result suggests populations of agents can accumulate knowledge the way search engines accumulate human artifacts, rather than repeating the same discoveries.","feed_headline":"Shared trajectory store raises agent success rates","feed_subtitle":"New agents complete more tasks in fewer steps by retrieving procedural knowledge produced by earlier agents in ALFWorld and WebArena.","key_machinery":"The Multi-Agent Transactive Memory framework for population-level storage and retrieval of agent-generated trajectories.","core_discovery":"Multi-Agent Transactive Memory stores trajectories contributed by producer agents in a shared repository so that consumer agents can retrieve relevant ones, producing higher downstream task success and fewer interaction steps in interactive environments without coordination or joint training.","pith_inferences":["Over time the repository could function as an accumulating library of solutions that grows with each new agent deployment.","Retrieval quality may depend on how well trajectory embeddings capture procedural similarity rather than surface features.","The same storage pattern could be tested on tasks outside simulated environments once suitable trajectory abstraction methods exist."],"forward_implications":["Producer agents contribute trajectories that later agents can apply to the same or similar tasks.","Task success rates rise in long-horizon interactive settings such as ALFWorld and WebArena.","Interaction length decreases without any requirement for agents to coordinate or train together.","The benefit holds across heterogeneous agent populations drawn from separate model families."],"fun_headline_variants":["MATM shares trajectories to improve agent task success","Trajectory retrieval reduces steps in interactive environments","Agents reuse trajectories from shared population memory","Transactive memory organizes agent generated procedural knowledge"],"cache_read_input_tokens":2112,"weakest_assumption_plain":"Agent-generated trajectories contain reusable procedural knowledge that matches and transfers directly to consumer agents from different instances or models without extra adaptation or filtering.","fun_headline_variants_meta":{"raw":{"variants":["MATM shares trajectories to improve agent task success","Trajectory retrieval reduces steps in interactive environments","Agents reuse trajectories from shared population memory","Transactive memory organizes agent generated procedural knowledge"]},"model":"grok-4.3","cost_usd":0.007344,"raw_usage":{"total_tokens":3349,"prompt_tokens":608,"num_sources_used":0,"completion_tokens":52,"cost_in_usd_ticks":73437000,"prompt_tokens_details":{"text_tokens":608,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":2689,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":608,"tokens_out":52,"duration_ms":21404,"temperature":1.0,"reasoning_tokens":2689,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-06-26T17:37:50.771028+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"A controlled test in which agents given retrieved trajectories show equal or lower success rates and equal or more steps than identical agents given no retrieval would falsify the performance claim.","supporting_citations":[],"review_version":1}