Pith. sign in

REVIEW 3 major objections 6 minor 46 references

LatticeMind: A Conflict-Aware Memory Primitive for Multi-Agent Systems

T0 review · 3 major / 6 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that treating contradiction as a write-time memory update, with explicit statuses and selective reconciliation, reaches 97 percent accuracy on a label-blind conflict benchmark, far above the 61 percent of the strongest…

desk verdict A well-specified memory primitive with a promising ConflictBank result, but the label-blind protocol may not remove the exact oracle cue it claims to, and the evaluation is too small to settle it. read the letter →

arxiv 2608.08236 v1 pith:UY6QPFI5 submitted 2026-08-08 cs.AI cs.CL

Heng Zhou , Lian Zhang , Yutao Fan , Tiancheng He , Siki Chen , Hejia Geng , Philip Torr , Zhenfei Yin This is my paper · ORCID
classification cs.AIcs.CL
keywords conflict-awarememorymulti-agentLLMknowledgeconflictswrite-timereconciliationstructuredstatuseslabel-blindevaluationsoftware-statetracking
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that multi-agent LLM systems fail at contradiction handling because they treat it as answer selection rather than persistent memory update. It introduces LatticeMind, a structured memory where each stored item carries an explicit status, cheap symbolic checks catch mechanical conflicts, and an LLM reconciler resolves only the remaining semantic disputes. On a label-blind public conflict benchmark with source names removed, LatticeMind reaches 0.97 accuracy versus 0.61 for the strongest answer-aggregation baseline, with all gaps significant at $p<10^{-6}$ by paired McNemar test. If the claim is right, the design goal shifts from better voting or judging to building memory that records which claim wins, which is contested, and why.

What carries the argument

The load-bearing mechanism is the write-time update loop with explicit per-item status. Each memory item is a tuple of key, content, evidence metadata, timestamp, and a status from proposed, confirmed, contested, and superseded; incoming claims are routed by a canonical key, a symbolic checker detects mechanical violations such as dependency cycles and resource-interval overlaps, and only unresolved semantic cases invoke the LLM reconciler. The reconciler separates credibility conflicts, where the winner is confirmed and the loser kept as contested or superseded, from coordination conflicts, where both candidates stay visible and planning reruns. This loop turns contradiction handling into a state transition on persistent memory rather than a one-shot selection among answers.

What would settle it

Rerun the label-blind evaluation on a version of the 75 examples where every textual evidence cue, including explicit citations, recency phrases, and self-described source kind, is stripped from the sources before the reconciler sees them. If LatticeMind's accuracy drops by much more than the 2-point drop already reported for removing role names, the 0.97 figure depends on cues that the label-blind protocol was meant to hide.

Watch

Extended reading notes

Core claim

The paper's central claim is that contradiction handling in multi-agent LLM systems should be a memory problem, not an answer-selection problem. LatticeMind is a structured memory in which each item carries a status drawn from proposed, confirmed, contested, and superseded; incoming claims are checked symbolically for mechanical conflicts such as dependency cycles and resource overlaps, and unresolved semantic conflicts are classified by an LLM reconciler as either credibility disputes, where one claim supersedes another, or coordination disputes, where both proposals remain visible and planning is rerun. On the label-blind protocol, where evidence sources appear under neutral names with no trust hints, this reaches 0.97 accuracy versus 0.63 for single-agent reading and 0.61 for the strongest answer-aggregation baseline, and removing either the checker or the reconciler costs 12 to 14 points. The paper reads this as evidence that persistent write-time conflict resolution, not answer aggregation, is the effective mechanism on this benchmark.

Load-bearing premise

The result depends on the assumption that replacing source names with neutral labels truly hides which source should be trusted; if the text still leaks authority through citations, recency markers, or self-descriptions, the reconciler could recover the hidden cue.

Editorial extensions

If this is right

  • If LatticeMind is correct, downstream agents read a resolved memory and do not relitigate the same conflict on later queries, so repeated similar questions no longer pay a fresh reconciliation cost.
  • The ablations imply the symbolic checker and the LLM reconciler are complementary: removing either one costs 12 to 14 accuracy points on the label-blind benchmark.
  • The same write-time rules transfer to long-horizon software-state tracking, with the benefit concentrated in provenance and temporal reasoning rather than in retrieving the current state.
  • The secondary planning results map the boundary of the approach: conflict-aware memory beats naive merge on three of four planning tasks, but does not replace iterative deliberation on tasks that reward search.
  • A learned version of the conflict and update operators transfers across unseen repository trajectories, while extraction remains the limiting step.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural extension the paper does not test is to run multi-agent debate with every round written into LatticeMind and the final answer read from the resolved memory, which would isolate whether persistent statuses help iterative protocols.
  • The coordination branch, which keeps both conflicting proposals visible and reruns planning, is introduced as a safety override without a validated benchmark; a dedicated resource-collision task would test whether this branch actually improves downstream decisions.
  • The learned-operator finding suggests the next bottleneck is extraction rather than conflict logic; a testable extension is training the extractor on more diverse raw repository styles and measuring whether end-to-end software-state scores rise without changing the update rules.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes LatticeMind, a structured memory primitive that handles contradictions at write time by maintaining explicit per-item statuses (PROPOSED, CONFIRMED, CONTESTED, SUPERSEDED), applying cheap symbolic conflict checks, and invoking selective LLM reconciliation for unresolved semantic conflicts. The central empirical claim is that on a 75-example label-blind ConflictBank evaluation, LatticeMind reaches 0.97 accuracy versus 0.61 for the strongest answer-aggregation baseline, with paired McNemar tests significant at p<10^-6. The paper also reports ablations, four secondary planning benchmarks with mixed results, a software-state application study, and a learned-operator study that localizes the remaining gap to extraction rather than conflict resolution.

Significance. If the ConflictBank result holds, the paper makes a useful contribution by reframing disagreement handling in multi-agent systems as a memory-update problem rather than an answer-selection problem. The paper ships per-sample outcomes, stand-alone statistical scripts, and honest discussion of limitations and negative results, which are strengths. However, the headline claim depends on the label-blind protocol actually removing the source-role oracle, and the current evidence does not establish that. The significance of the 0.97 versus 0.61 gap is therefore uncertain pending additional controls that isolate the evidence-type tags and the persistence of the memory mechanism.

major comments (3)
  1. [A.9, §4.1, Table 1] The load-bearing premise of the label-blind protocol is that it removes the source-role oracle, but the reconciler is explicitly described as scoring a claim from evidence-type tags inferred from each source's text, including explicit citations, recency markers, and self-described source kind. These are precisely the surface cues that correlate with ConflictBank's original roles of authoritative, alternative, and recent source. The protocol renames sources to Source A/B/C and removes trust hints from reader prompts, but it does not remove or neutralize these text-derived tags. The one experiment intended to strip evidence labels is the last row of Table 1, and the paper itself acknowledges that it is a repeat control, so no experiment directly tests the contribution of the tags. Table 8 removes role names from the reader prompt only, which is consistent with LatticeMind relying on evidence tags that were never removed. I would like to see (i) an ablation that removes or shuffles evidence-type tags from the reconciler input, and (ii) an answer-level baseline that performs the same evidence-tag-weighted reconciliation in one shot. Without these, the 0.97 versus 0.61 gap may reflect a re-coding of the source-authority cue rather than the write-time memory mechanism.
  2. [§4.1, Table 9] The paper does not specify how the 75 ConflictBank examples were sampled, from which split, or why the ablation in Table 1 uses 50 samples. This matters because the headline accuracy, confidence intervals, and McNemar tests are all conditional on this subset. Please report the sampling procedure, the seed, the composition of the subset (for example, conflict types and evidence distributions), and ideally results on the full ConflictBank evaluation set or across multiple seeds.
  3. [§4.1, A.9] The comparison conflates the memory structure with the presence of an additional LLM reconciliation step. LatticeMind invokes an LLM reconciler on unresolved conflicts, while the answer-level baselines use fixed aggregation rules; the Judge baseline selects among answers rather than performing evidence-weighted reconciliation over the same per-source extractions and tags. To support the claim that persistent write-time memory is the effective mechanism, the paper should include a one-shot LLM judge or reconciler baseline that receives the same three per-source extractions and the same evidence-type tags and is asked to return an answer immediately. If such a baseline approaches 0.97, the persistence of the memory is not what drives the gain.
minor comments (6)
  1. [§4.1] The phrase 'drops any trust hint from the reader prompts' should be clarified to state whether the evidence-type tags presented to the reconciler are considered trust hints, and if not, why they are treated differently from the removed source-name hints.
  2. [Table 1, A.9] The conflict-rate metric is 0.00 for Full and 0.02 for Full repeat, yet the text in A.9 says both configurations report 3.0 ungrouped pairwise violations per example. Please define the metric precisely and explain the source of the 0.02 discrepancy, for example whether it arises from grouping or from answer-time rendering.
  3. [A.1] Because the raw prompt and response logs cannot be redistributed due to ConflictBank's license, consider releasing a redacted version or per-example feature hashes so that the label-blind protocol can be independently audited without the raw evidence passages.
  4. [Table 2, A.5] The 'Repo bridge (v2)' row in Table 2 is not described in the main text; please add a reference to Section A.5 and state explicitly how v2 differs from the earlier bridge, since the harder bridge changes the relative ranking of methods.
  5. [§4.5] The sentence 'The harder repo bridge is challenging for the current instantiation (LLM-MERGE 0.61 vs. STATEMEMORY 0.51)' should be expanded to note clearly that this is a case where the structured schema underperforms a strong summarizer, as is acknowledged in the discussion, so that the reader does not misread it as a typo.
  6. [A.9] The paper states that source order is preserved across examples for replay determinism and that the reconciler is content-driven; please report whether any auxiliary experiment randomized source order, since position could still be a confound even if the policy is nominally stateless.

Circularity Check

2 steps flagged · score 4.0 of 10

Label-blind protocol may re-admit the source-role oracle through evidence-type tags; the intended evidence-label ablation reduces to a repeat run.

  1. renaming known result [Appendix A.9 'ConflictBank agents and prompts'; Section 4.1 'Label-blind protocol']
    "Crucially, neither the extractor nor the reconciler receives any role label or source-name hint indicating which source is the authoritative, alternative, or recent one. ... it scores a claim from the evidence-type tags inferred from each source's text (e.g., explicit citations, recency markers, or self-described source kind), not from per-example position."

    The label-blind protocol is defined as removing role names and trust hints from reader prompts, but the reconciler's scoring function is fed evidence-type tags inferred from the source text, including 'self-described source kind' and recency markers. In ConflictBank the confound to be removed is exactly the source kind (authoritative/alternative/recent). The tags are thus the same oracle cue under a new surface, so the 0.97 accuracy can be produced by the cue the protocol claims to hide. Table 8 only strips role names from the reader prompt, not from the tag input, so the paper's assertion that 'the gain is not the evidence-tag heuristic itself' is not tested by the design.

  2. other [Section 4.3 'Ablation on Contradiction Handling', Table 1 note]
    "The last row deserves an explicit caveat. It was originally intended as an evidence-label ablation, stripping role names such as 'authoritative source' from the evidence attached to each finding. Under the label-blind protocol those role names are already absent, so the configuration reduces to a second run of the full system."

    The one experiment that would directly isolate whether evidence labels drive the result was intended to strip role names from the evidence attached to each finding. By the paper's own admission, under the label-blind protocol this configuration reduces by construction to a second run of the full system, so it no longer removes the cue from the reconciler's tag input. The authors re-purpose the row as a noise control, but this is an explicit acknowledgement that the direct test of the load-bearing premise (that the tags do not carry the oracle) is absent.

full rationale

Most of the paper is an empirical benchmark comparison rather than a formal derivation, so the circularity burden is low. The memory-update rules, the software-state evidence score, and the learned-operator study do not define their outputs in terms of their targets; the LoRA study is a standard leave-one-out transfer check, and the author self-citations (e.g., Zhou et al. 2025, 2026a, 2026b; Zhang et al. 2026) are background context rather than load-bearing evidence. The one step that approaches circularity is the label-blind ConflictBank protocol. The paper's own Appendix A.9 states that the reconciler scores claims from evidence-type tags inferred from each source's text, including 'self-described source kind' and recency markers, while the protocol removes only role names from the reader prompts. Since ConflictBank's original oracle cue is the source kind (authoritative/alternative/recent), these tags can re-encode the very label the protocol claims to hide. The intended evidence-label ablation 'reduces to a second run of the full system' by the paper's own admission, so the direct test of the leakage hypothesis is missing. This makes the headline 0.97 vs 0.61 comparison partially self-supporting if the tags carry the role cue. I weigh this as partial circularity (4) rather than higher because the result is an external benchmark comparison with released per-sample runners, and the concern is disclosed in the paper rather than a formal equivalence.

Assumptions & free parameters 1 free parameters · 3 assumptions · 1 invented entities

The central claim rests primarily on the ConflictBank evaluation. The free parameters are the hand-set evidence weights that drive supersession; the main axioms are the effectiveness of label-blinding and the stability of a single LLM reconciliation call. The invented status field has no isolated measurement. No mathematical derivation is involved, so the ledger is dominated by experimental assumptions rather than formal axioms.

free parameters (1)
  • evidence-type weights (Table 7) = code-change 60; incident-hotfix 60; config-observation 45; runtime-observation 40; branch-experiment 25; human-note…
    Hand-set numeric weights define which claim supersedes which in the software-state resolver and the general evidence comparison; no fitting or sensitivity analysis is reported, and the +40 git bonus dominates ordinary type differences.
assumptions (3)
  • domain assumption The reconciler's evidence-type inference from source text (citations, recency markers, self-described source kind) recovers the intended credibility ordering after label blinding.
    Appendix A.9 states the reconciler scores claims from evidence-type tags inferred from each source's text; if these cues still encode the original authoritative/alternative/recent roles, the 0.97 result can be explained by label leakage rather than by the memory update.
  • domain assumption A single-pass LLM reconciliation call yields a stable and correct status transition for each conflict.
    The method relies on the hosted LLM to classify conflicts as credibility or coordination and to pick winners; repeated-run consistency is checked with only one repeated run, and no cross-model or decoding-seed variation is reported.
  • ad hoc to paper The four-state status machine (PROPOSED, CONFIRMED, CONTESTED, SUPERSEDED) with central orchestrator transitions is an adequate model of trust over time for these tasks.
    The status semantics and transition rules are introduced in Algorithm 1 and A.9 without a formal account of how statuses interact with retrieval or how they generalize beyond the evaluated tasks.
invented entities (1)
  • Per-item status field with states PROPOSED, CONFIRMED, CONTESTED, SUPERSEDED
    purpose: Makes conflict resolution a persistent write-time memory operation and renders a current trusted view.
    The status is an internal software construct; its contribution is only measured end-to-end through task accuracy, with no isolated experiment that varies the status field while holding prompts and resolver fixed.

how reviews work

0 comments
Cite this review

Pith. "Pith review of LatticeMind: A Conflict-Aware Memory Primitive for Multi-Agent Systems." pith.science (2026). https://pith.science/paper/UY6QPFI5

@misc{pith2026260808236,
  author       = {Pith},
  title        = {Pith review of: LatticeMind: A Conflict-Aware Memory Primitive for Multi-Agent Systems},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UY6QPFI5}},
  note         = {Machine review of arXiv:2608.08236}
}
abstract

Multi-agent LLM systems often fail not for lack of candidate answers, but because they have no persistent mechanism for deciding which incompatible claim should currently be trusted. Majority vote, debate, and judge-based selection choose an output without recording which claim wins, which is contested, or why a later update supersedes it. We present \term{LatticeMind}, a conflict-aware structured memory that handles contradiction at write time. It maintains explicit item status, applies cheap symbolic conflict checks, and invokes LLM reconciliation only for unresolved semantic cases. On a label-blind ConflictBank evaluation that removes source-name hints, LatticeMind reaches 0.97 accuracy versus 0.61 for the strongest aggregation baseline, with the gap significant at $p<10^{-6}$ by paired McNemar test. Ablations show that removing the checker or the reconciler costs 12 to 14 points. On four secondary planning benchmarks the picture is mixed: LatticeMind beats naive merge on three of four, but does not replace deliberation methods on tasks rewarding iterative search.

Figures

Figures reproduced from arXiv: 2608.08236 by the authors.

Figure 1
Figure 1. LatticeMind overview on an approval-policy example. Agents write structured findings into shared [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Label-blind ConflictBank accuracy across [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. compares LatticeMind against NO￾MERGE, the most apples-to-apples baseline since both methods use the same multi-agent pool and NaturalPlan Calendar PlanBench Travel Planner REALM JSSP 0.00 0.25 0.50 0.75 1.00 Score 0.67 0.47 0.87 0.10 0.80 +0.13 0.60 +0.13 0.80 -0.07 0.20 +0.10 No-Merge LatticeMind [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Per-family scores on the budgeted long-noise [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 6
Figure 6. Figure 6: Representative patch-guardrails extraction example. The direct extractor drifts into the wrong entity/slot family and normalizes to an empty claim set, while the two-stage extractor recovers the correct patch_runtime/approvals frame and only misses on value specificity…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

46 extracted references · 9 canonical work pages

  1. [1]

    Prateek Chhikara, Dev Khant, Saket Aryan, Taranjeet Singh, and Deshraj Yadav. 2025. Mem0 : Building production-ready AI agents with scalable long-term memory. arXiv preprint arXiv:2504.19413

    arXiv 2025

  2. [2]

    Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. 2023. https://arxiv.org/abs/2305.14314 QLoRA : Efficient finetuning of quantized LLMs . In Advances in Neural Information Processing Systems (NeurIPS)

    arXiv 2023

  3. [3]

    Tenenbaum, and Igor Mordatch

    Yilun Du, Shuang Li, Antonio Torralba, Joshua B. Tenenbaum, and Igor Mordatch. 2024. https://arxiv.org/abs/2305.14325 Improving factuality and reasoning in language models through multiagent debate . In International Conference on Machine Learning (ICML)

    arXiv 2024

  4. [4]

    Yukang Feng, Jianwen Sun, Zelai Yang, Jiaxin Ai, Chuanhao Li, Zizhen Li, Fanrui Zhang, Kang He, Rui Ma, Jifan Lin, Jie Sun, Yang Xiao, Sizhuo Zhou, Wenxiao Wu, Yiming Liu, Pengfei Liu, Yu Qiao, Shenglin Zhang, and Kaipeng Zhang. 2026. https://doi.org/10.48550/arXiv.2602.14337 Longcli-bench: A preliminary benchmark and study for long-horizon agentic progra...

    work page doi:10.48550/arxiv.2602.14337 2026

  5. [5]

    Longling Geng and Edward Y. Chang. 2025. REALM-Bench : A benchmark for evaluating multi-agent systems on real-world, dynamic planning and scheduling tasks. arXiv preprint arXiv:2502.18836

    arXiv 2025

  6. [6]

    Chawla, Olaf Wiest, and Xiangliang Zhang

    Taicheng Guo, Xiuying Chen, Yaqi Wang, Ruidi Chang, Shichao Pei, Nitesh V. Chawla, Olaf Wiest, and Xiangliang Zhang. 2024. https://arxiv.org/abs/2402.01680 Large language model based multi-agents: A survey of progress and challenges . In Proceedings of the 33rd International Joint Conference on Artificial Intelligence (IJCAI)

    arXiv 2024

  7. [7]

    Bernal Jim \'e nez Guti \'e rrez, Yiheng Shu, Yu Gu, Michihiro Yasunaga, and Yu Su. 2024. https://arxiv.org/abs/2405.14831 HippoRAG : Neurobiologically inspired long-term memory for large language models . In Advances in Neural Information Processing Systems (NeurIPS)

    arXiv 2024

  8. [8]

    Sirui Hong, Mingchen Zhuge, Jonathan Chen, Xiawu Zheng, Yuheng Cheng, Jinlin Wang, Ceyao Zhang, Zili Wang, Steven Ka Shing Yau, Zijuan Lin, Liyang Zhou, Chenyu Ran, Lingfeng Xiao, Chenglin Wu, and J \"u rgen Schmidhuber. 2024. https://arxiv.org/abs/2308.00352 MetaGPT : Meta programming for a multi-agent collaborative framework . In International Conferenc...

    arXiv 2024

Show all 46 references
  1. [9]

    Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen

    Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. https://arxiv.org/abs/2106.09685 LoRA : Low-rank adaptation of large language models . In International Conference on Learning Representations (ICLR)

    2022 arXiv

  2. [10]

    Yuanzhe Hu, Yu Wang, and Julian McAuley. 2025 a . Evaluating memory in LLM agents via incremental multi-turn interactions. arXiv preprint arXiv:2507.05257

    2025 arXiv

  3. [11]

    Yuyang Hu, Shichun Liu, Yanwei Yue, Guibin Zhang, Boyang Liu, Fangyi Zhu, Jiahang Lin, Honglin Guo, Shihan Dou, Zhiheng Xi, Senjie Jin, Jiejun Tan, Yanbin Yin, Jiongnan Liu, Zeyu Zhang, Zhongxiang Sun, Yutao Zhu, Hao Sun, Boci Peng, and 28 others. 2025 b . https://doi.org/10.4...

    2025 doi arXiv

  4. [12]

    Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik R

    Carlos E. Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik R. Narasimhan. 2024. https://arxiv.org/abs/2310.06770 SWE-bench : Can language models resolve real-world GitHub issues? In International Conference on Learning Representations (ICLR)

    2024 arXiv

  5. [13]

    u ttler, Mike Lewis, Wen-tau Yih, Tim Rockt \

    Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich K \"u ttler, Mike Lewis, Wen-tau Yih, Tim Rockt \"a schel, Sebastian Riedel, and Douwe Kiela. 2020. https://arxiv.org/abs/2005.11401 Retrieval-augmented generation for knowl...

    2020 arXiv

  6. [14]

    Guohao Li, Hasan Abed Al Kader Hammoud, Hani Itani, Dmitrii Khizbullin, and Bernard Ghanem. 2023. https://arxiv.org/abs/2303.17760 CAMEL : Communicative agents for ``mind'' exploration of large language model society . In Advances in Neural Information Processing Systems (NeurIPS)

    2023 arXiv

  7. [15]

    Adyasha Maharana, Dong-Ho Lee, Sergey Tulyakov, Mohit Bansal, Francesco Barbieri, and Yuwei Fang. 2024. https://doi.org/10.18653/v1/2024.acl-long.747 Evaluating very long-term conversational memory of LLM agents . In Proceedings of the 62nd Annual Meeting of the Association fo...

    2024 doi

  8. [16]

    Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. 2022. https://arxiv.org/abs/2202.05262 Locating and editing factual associations in GPT . In Advances in Neural Information Processing Systems (NeurIPS)

    2022 arXiv

  9. [17]

    Patil, Ion Stoica, and Joseph E

    Charles Packer, Sarah Wooders, Kevin Lin, Vivian Fang, Shishir G. Patil, Ion Stoica, and Joseph E. Gonzalez. 2023. MemGPT : Towards LLMs as operating systems. arXiv preprint arXiv:2310.08560

    2023 arXiv

  10. [18]

    O'Brien, Carrie J

    Joon Sung Park, Joseph C. O'Brien, Carrie J. Cai, Meredith Ringel Morris, Percy Liang, and Michael S. Bernstein. 2023. https://arxiv.org/abs/2304.03442 Generative agents: Interactive simulacra of human behavior . In Proceedings of the 36th Annual ACM Symposium on User Interfac...

    2023 arXiv

  11. [19]

    Chen Qian, Wei Liu, Hongzhang Liu, Nuo Chen, Yufan Dang, Jiahao Li, Cheng Yang, Weize Chen, Yusheng Su, Xin Cong, Juyuan Xu, Dahai Li, Zhiyuan Liu, and Maosong Sun. 2024. https://arxiv.org/abs/2307.07924 ChatDev : Communicative agents for software development . In Proceedings ...

    2024 arXiv

  12. [20]

    Qwen Team . 2024. Qwen2.5 technical report. arXiv preprint arXiv:2412.15115

    2024 arXiv

  13. [21]

    Preston Rasmussen, Pavlo Paliychuk, Travis Beauvais, Jack Ryan, and Daniel Chalef. 2025. Zep : A temporal knowledge graph architecture for agent memory. arXiv preprint arXiv:2501.13956

    2025 arXiv

  14. [22]

    Marc Shapiro, Nuno Pregui c a, Carlos Baquero, and Marek Zawirski. 2011. Conflict-free replicated data types. In Stabilization, Safety, and Security of Distributed Systems (SSS), volume 6976 of Lecture Notes in Computer Science. Springer

    2011

  15. [23]

    Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. 2023. https://arxiv.org/abs/2303.11366 Reflexion: Language agents with verbal reinforcement learning . In Advances in Neural Information Processing Systems (NeurIPS)

    2023 arXiv

  16. [24]

    Zhaochen Su, Jun Zhang, Xiaoye Qu, Tong Zhu, Yanshu Li, Jiashuo Sun, Juntao Li, Min Zhang, and Yu Cheng. 2024. https://arxiv.org/abs/2408.12076 ConflictBank : A benchmark for evaluating the influence of knowledge conflicts in LLM . In Advances in Neural Information Processing ...

    2024 arXiv

  17. [25]

    Sumers, Shunyu Yao, Karthik Narasimhan, and Thomas L

    Theodore R. Sumers, Shunyu Yao, Karthik Narasimhan, and Thomas L. Griffiths. 2024. https://arxiv.org/abs/2309.02427 Cognitive architectures for language agents . Transactions on Machine Learning Research (TMLR)

    2024 arXiv

  18. [26]

    Haoran Tan, Zeyu Zhang, Chen Ma, Xu Chen, Quanyu Dai, and Zhenhua Dong. 2025. Membench: Towards more comprehensive evaluation on the memory of llm-based agents. In Findings of the Association for Computational Linguistics: ACL 2025, pages 19336--19352. Association for Computat...

    2025

  19. [27]

    Karthik Valmeekam, Matthew Marquez, Alberto Olmo, Sarath Sreedharan, and Subbarao Kambhampati. 2023. https://arxiv.org/abs/2206.10498 PlanBench : An extensible benchmark for evaluating large language models on planning and reasoning about change . In Advances in Neural Informa...

    2023 arXiv

  20. [28]

    Xu, Xiangru Tang, Mingchen Zhuge, Jiayi Pan, Yueqi Song, Bowen Li, Jaskirat Singh, Hoang H

    Xingyao Wang, Boxuan Li, Yufan Song, Frank F. Xu, Xiangru Tang, Mingchen Zhuge, Jiayi Pan, Yueqi Song, Bowen Li, Jaskirat Singh, Hoang H. Tran, Fuqiang Li, Ren Ma, Mingzhang Zheng, Bill Qian, Yanjun Shao, Niklas Muennighoff, Yizhe Zhang, Binyuan Hui, and 5 others. 2024. OpenHa...

    2024 arXiv

  21. [29]

    Le, Ed H

    Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V. Le, Ed H. Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2023. https://arxiv.org/abs/2203.11171 Self-consistency improves chain of thought reasoning in language models . In International Conference on Learning Represen...

    2023 arXiv

  22. [30]

    Yu Wang and Xi Chen. 2025. MIRIX : Multi-agent memory system for LLM -based agents. arXiv preprint arXiv:2507.07957

    2025 arXiv

  23. [31]

    Di Wu, Hongwei Wang, Wenhao Yu, Yuwei Zhang, Kai-Wei Chang, and Dong Yu. 2025. https://proceedings.iclr.cc/paper_files/paper/2025/hash/d813d324dbf0598bbdc9c8e79740ed01-Abstract-Conference.html Longmemeval: Benchmarking chat assistants on long-term interactive memory . In Inter...

    2025

  24. [32]

    White, Doug Burger, and Chi Wang

    Qingyun Wu, Gagan Bansal, Jieyu Zhang, Yiran Wu, Beibin Li, Erkang Zhu, Li Jiang, Xiaoyun Zhang, Shaokun Zhang, Jiale Liu, Ahmed Hassan Awadallah, Ryen W. White, Doug Burger, and Chi Wang. 2023. AutoGen : Enabling next-gen LLM applications via multi-agent conversation. arXiv p...

    2023 arXiv

  25. [33]

    Zhiheng Xi, Wenxiang Chen, Xin Guo, Wei He, Yiwen Ding, Boyang Hong, Ming Zhang, Junzhe Wang, Senjie Jin, Enyu Zhou, Rui Zheng, Xiaoran Fan, Xiao Wang, Limao Xiong, Yuhao Zhou, Weiran Wang, Changhao Jiang, Yicheng Zou, Xiangyang Liu, and 10 others. 2023. The rise and potential...

    2023 arXiv

  26. [34]

    Chunqiu Steven Xia, Yinlin Deng, Soren Dunn, and Lingming Zhang. 2024. Agentless : Demystifying LLM -based software engineering agents. arXiv preprint arXiv:2407.01489

    2024 arXiv

  27. [35]

    Jian Xie, Kai Zhang, Jiangjie Chen, Tinghui Zhu, Renze Lou, Yuandong Tian, Yanghua Xiao, and Yu Su. 2024. https://arxiv.org/abs/2402.01622 TravelPlanner : A benchmark for real-world planning with language agents . In International Conference on Machine Learning (ICML)

    2024 arXiv

  28. [36]

    Rongwu Xu, Zehan Qi, Zhijiang Guo, Cunxiang Wang, Hongru Wang, Yue Zhang, and Wei Xu. 2024. https://arxiv.org/abs/2403.08319 Knowledge conflicts for LLMs : A survey . In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 8541--8565

    2024 arXiv

  29. [37]

    Wujiang Xu, Zujie Liang, Kai Mei, Hang Gao, Juntao Tan, and Yongfeng Zhang. 2025. A-MEM : Agentic memory for LLM agents. arXiv preprint arXiv:2502.12110

    2025 arXiv

  30. [38]

    Jimenez, Alexander Wettig, Kilian Lieret, Shunyu Yao, Karthik Narasimhan, and Ofir Press

    John Yang, Carlos E. Jimenez, Alexander Wettig, Kilian Lieret, Shunyu Yao, Karthik Narasimhan, and Ofir Press. 2024. SWE-agent : Agent-computer interfaces enable automated software engineering. In Advances in Neural Information Processing Systems (NeurIPS)

    2024

  31. [39]

    Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. 2023. https://arxiv.org/abs/2210.03629 ReAct : Synergizing reasoning and acting in language models . In International Conference on Learning Representations (ICLR)

    2023 arXiv

  32. [40]

    Guibin Zhang, Hejia Geng, Xiaohang Yu, Zhenfei Yin, Zaibin Zhang, Zelin Tan, Heng Zhou, Zhongzhi Li, Xiangyuan Xue, Yijiang Li, Yifan Zhou, Yang Chen, Chen Zhang, Yutao Fan, Zihu Wang, Songtao Huang, Francisco Piedrahita-Velez, Yue Liao, Hongru Wang, and 6 others. 2026. The la...

    2026

  33. [41]

    Qinggang Zhang, Zhishang Xiang, Yilin Xiao, Le Wang, Junhui Li, Xinrun Wang, and Jinsong Su. 2025. FaithfulRAG : Fact-level conflict modeling for context-faithful retrieval-augmented generation. arXiv preprint arXiv:2506.08938

    2025 arXiv

  34. [42]

    Le, Ed H

    Huaixiu Steven Zheng, Swaroop Mishra, Hugh Zhang, Xinyun Chen, Minmin Chen, Azade Nova, Le Hou, Heng-Tze Cheng, Quoc V. Le, Ed H. Chi, and Denny Zhou. 2024. NATURAL PLAN : Benchmarking LLMs on natural language planning. arXiv preprint arXiv:2406.04520

    2024 arXiv

  35. [43]

    Xing, Hao Zhang, Joseph E

    Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric P. Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. 2023. https://arxiv.org/abs/2306.05685 Judging LLM -as-a-judge with MT-Bench and chatbot arena . ...

    2023 arXiv

  36. [44]

    Heng Zhou, Hejia Geng, Xiangyuan Xue, Li Kang, Yiran Qin, Zhiyong Wang, Zhenfei Yin, and Lei Bai. 2025. RESO : A reward-driven self-organizing LLM -based multi-agent system for reasoning tasks. In Proceedings of the Conference on Empirical Methods in Natural Language Processin...

    2025

  37. [45]

    Heng Zhou, Zelin Tan, Zhemeng Zhang, Yutao Fan, Yibing Lin, Li Kang, Xiufeng Song, Rui Li, Songtao Huang, Ao Yu, Yuchen Fan, Yanxu Chen, Kaixin Xu, Xiaohong Liu, Yiran Qin, Philip Torr, Chen Zhang, and Zhenfei Yin. 2026 a . Select-then-solve: Paradigm routing as inference-time...

    2026

  38. [46]

    Heng Zhou, Ao Yu, Yuchen Fan, Jianing Shi, Li Kang, Hejia Geng, Yongting Zhang, Yutao Fan, Yuhao Wu, Tiancheng He, Yiran Qin, Lei Bai, and Zhenfei Yin. 2026 b . LiveSearchBench : An automatically constructed benchmark for retrieval and reasoning over dynamic knowledge. In ICML...

    2026

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.