Pith. sign in

REVIEW 5 major objections 6 minor 49 references

Evo-MARL: Co-Evolutionary Multi-Agent Reinforcement Learning for Internalized Safety

T0 review · 5 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Training every task agent to resist attacks, without an external guard, reduces attack success rates by up to 22% while raising reasoning accuracy by up to 5%.

desk verdict A genuinely new MARL recipe for internalizing safety in LLM agents, but the shared LLaMA-Guard reward/eval metric makes the headline ASR numbers provisional. read the letter →

arxiv 2508.03864 v2 pith:57DJGWQY submitted 2025-08-05 cs.AI

Zhenyu Pan , Yiting Zhang , Yutong Zhang , Jianshu Zhang , Haozheng Luo , Yuwei Han , Dennis Wu , Hong-Yu Chen
show 3 more authors
Philip S. Yu Manling Li Han Liu
This is my paper
classification cs.AI
keywords multi-agentsystemslargelanguagemodelssafetyreinforcementlearningadversarialtrainingevolutionarysearchjailbreakdefenseGRPO
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether a multi-agent system built from language models can be made safer from within, rather than by bolting on a separate safety agent. The authors propose Evo-MARL, which trains every task agent with a shared multi-agent reinforcement-learning objective to both perform its job and resist adversarial prompts, while a co-evolving pool of attacks keeps applying pressure. They report that on red-team benchmarks the trained systems reduce attack success rates by up to 22% and on reasoning and writing tasks improve accuracy by up to 5%, so safety and utility improve together. If the result holds, it would mean defense can be internalized into the agents themselves, avoiding a single point of failure and extra guard modules.

What carries the argument

The central mechanism is a chain-structured multi-agent system whose defenders share one policy, conditioned by role, trained with Group Relative Policy Optimization (GRPO), while a separate evolutionary loop mutates attack prompts and keeps the ones that succeed. The co-evolution is what does the work: attackers are excluded from RL to avoid conflicting gradients, so the evolutionary search supplies the adaptive pressure that keeps defenders from memorizing a fixed set of jailbreaks. The reward function ties defense to utility by paying +1 for a safe final response and +0.5 for a correct one.

What would settle it

Re-run the evaluation with a different safety judge—human raters or a classifier not used in training—and compare attack success rates on the trained versus untrained systems; if the gap shrinks or vanishes, the reported gains are an artifact of optimizing LLaMA-Guard-3-8B's judgments rather than internalized robustness.

Watch

Extended reading notes

Core claim

Evo-MARL claims that safety can be internalized in a multi-agent LLM system by jointly optimizing all task agents. In training, a chain-structured system of three role-specialized agents is repeatedly exposed to attacks: a randomly chosen agent is compromised via indirect prompt injection, and the other agents must detect, discard, or purify the malicious content while still producing the correct answer. Defenders share parameters and are optimized with GRPO, with rewards of +1 for a final response that LLaMA-Guard-3-8B judges safe and +0.5 for correctness; the attacker is not RL-trained but evolves through mutation, crossover, and selection based on attack success. On JailBreakV, HarmBench, and MultiJail, the trained 1.5B-parameter MAS lowers attack success rate by up to 22%, beats untrained 3B-parameter MAS on safety, and reaches safety close to or above an untrained 7B-parameter MAS; on MATH and Creative Writing, accuracy rises by up to 5 percentage points. The authors conclude that method-level training, rather than model scaling, is the effective lever for MAS safety.

Load-bearing premise

The whole result rests on LLaMA-Guard-3-8B's harmfulness judgment being a faithful measure of real safety, because that same judgment is both the training reward and the evaluation metric.

Editorial extensions

If this is right

  • External guard modules become unnecessary: every task agent is itself a defender, so compromising one agent no longer by itself collapses system safety.
  • Safety can be bought with training rather than compute: a 1.5B-parameter trained MAS can be safer than untrained 3B- and 7B-based systems on several benchmarks.
  • The safety-helpfulness tradeoff is not inevitable: the same reward loop that suppresses attacks raises accuracy on MATH and Creative Writing.
  • The co-evolutionary attack pool keeps the defense from over-fitting to static jailbreaks, since attack prompts are mutated and selected for current success.
  • The training recipe transfers across text-only and multimodal MAS and across chain and hierarchical topologies, as the evaluation includes both.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial extension: the evaluation attacks in JailBreakV, HarmBench, and MultiJail are close in form to the injections used in training, so the method's advertised robustness is measured on near-distribution attacks; a held-out adaptive attacker would test whether the internalized defense actually generalizes.
  • Editorial extension: because the same safety judge (LLaMA-Guard-3-8B) is both the training reward and the evaluation metric, the reported 22% safety gain could be reproduced or falsified by swapping in a different judge or human raters at test time without changing the training pipeline.
  • Editorial extension: the architecture trains against a chain of three agents with shared parameters, so the learned defensive policy may transfer to other topologies such as tree or mesh agent networks; that is a plausible but untested consequence.
  • Editorial extension: an ablation that removes the safety reward while keeping the correctness reward would reveal how much of the 5% accuracy gain comes from GRPO's general reasoning improvement rather than from the safety objective; the paper does not isolate this.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper proposes Evo-MARL, a multi-agent reinforcement learning framework that trains all task agents in a chain-structured MAS to resist jailbreak and injection attacks instead of relying on an external guard module. Attack prompts are maintained in an evolving population via mutation, crossover, and fitness-based selection driven by attack success; defenders share parameters and are trained with GRPO using rewards for final-response safety (judged by LLaMA-Guard-3-8B) and correctness. Experiments on JailBreakV-28K (280-sample subset), HarmBench multimodal split, MultiJail English split, MATH (100 random samples), and Creative Writing report ASR reductions up to roughly 21 percentage points and accuracy gains up to 5 points on MATH. The paper argues that these results demonstrate jointly improved safety and utility without external safety modules.

Significance. The direction is timely and valuable: internalizing safety into every task agent with shared-policy MARL is a plausible alternative to centralized guard modules and may reduce single-point-of-failure risks. The co-evolutionary attacker pool is a reasonable mechanism for avoiding overfitting to a static attack set. If the reported improvements survive a more controlled evaluation—an independent safety metric, guard-module baselines, ablations of the evolutionary component, and repeated-seed statistics—the framework would be a solid contribution to LLM multi-agent safety. At present the paper is primarily empirical and the evidence is not yet conclusive; no code release is mentioned, and the central safety metric is shared between training and evaluation, which limits what the current numbers can establish.

major comments (5)
  1. [§3.1, §4.1] The safety signal used for training and the safety metric used for evaluation are the same LLaMA-Guard-3-8B classifier: §3.1 awards +1 when the final response is "deemed safe", and §4.1 states that ASR is computed with responses' "harmfulness evaluated by LLaMA-Guard-3-8B". GRPO can therefore improve the reported ASR by exploiting the classifier's decision surface (e.g., adding disclaimers, hedging, or formulaic refusals) without genuinely increasing robustness. As a concrete test, the authors should re-evaluate the final checkpoints with at least one independent safety evaluator not used in training (a different classifier, human annotation on a sample, or a refusal-quality protocol) and report agreement/disagreement with LLaMA-Guard. Without this, the headline "reduces attack success rates by up to 22%" does not establish internalized safety.
  2. [Table 1, §4.1] The comparative evidence in Table 1 is statistically thin. Each cell is a single point estimate with no error bars, no repeated seeds, and no significance tests. The JailBreakV evaluation uses only 280 samples and MATH uses 100; the sizes of the HarmBench, MultiJail, and Creative Writing splits are not reported. Several reported improvements are small (e.g., MultiJail 17% to 13%, Creative Writing 8.2% to 8.6%) and could be sampling noise under stochastic decoding. The claim of "consistent safety improvements across all tasks and model scales" requires at least a small number of training/evaluation runs with variance reporting or bootstrap confidence intervals to be credible.
  3. [§4.2, Table 1] The experiments do not isolate the two claimed mechanisms. The contribution claims both (i) internalizing defense into every task agent instead of an external guard module and (ii) co-evolution of attacks and defenses. Table 1 only compares trained MAS against untrained MAS of different scales. There is no MAS with a guard agent or peer-review defense (e.g., the approaches cited in §2), no ablation with a fixed/static attack pool instead of the evolutionary pool, and no ablation of the correctness-reward weighting. Without these comparisons, the improvement cannot be attributed to Evo-MARL specifically, as opposed to any RL fine-tuning on the red-team distribution.
  4. [§4.1] The multi-modal evaluation is under-specified. JailBreakV-28K and HarmBench include image-based attacks, but all agents are said to be Qwen2.5-1.5B/3B-Instruct, which are text-only. The hierarchical setup mentions a "jailbreak-prone multimodal agent" but does not identify its model, how image inputs are handled, or how the training-time chain (which appears text-based) transfers to this setup. Since the metric only counts samples that first compromise the multimodal agent, the reported ASR depends on an unspecified first-stage model. Please clarify the exact architecture for both settings, including which components are trained.
  5. [Eq. (1), §3.2, §4.1] Reproducibility details are missing for the central training procedure. The manuscript does not specify the mutation/crossover operators, population size, selection mechanism, or number of generations for the evolutionary attack pool; the GRPO hyperparameters (learning rate, number of samples G, KL penalty coefficient, rollout size) are not given; and Eq. (1) appears with corrupted symbols. For an empirical paper whose main deliverable is a training method, these details are necessary and should be reported in a table or appendix.
minor comments (6)
  1. [Abstract, Table 1] The abstract's "up to 22%" reduction does not clearly correspond to any cell in Table 1: the largest absolute drop is from 69% to 48% on HarmBench (21 percentage points), and the relative reduction for that row is about 30%. Please reconcile the headline number with the table.
  2. [Table 1] The MAS-7B row should be labeled "Untrained" for consistency with the MAS-1.5B and MAS-3B rows, and the fact that no 7B-trained model is reported should be stated explicitly.
  3. [Figure 2] The qualitative examples in Figure 2 would benefit from a short paragraph describing what specifically changed after training; currently they are presented without analysis.
  4. [§4.2] The wording "boosting accuracy by up to 5%" should say "5 percentage points" to avoid confusion with relative gains.
  5. [References] There is a typo in the reference list: "tse Huang et al." should be "Jen-tse Huang et al." in the citation in §2.
  6. [General] The paper does not state whether code or checkpoints will be released; please add an availability statement.

Circularity Check

1 steps flagged · score 6.0 of 10

The shared LLaMA-Guard reward and evaluation metric makes reported ASR reductions a measure of training-objective optimization rather than independent safety.

  1. fitted input called prediction [Section 3.1 (reward definition) and Section 4.1 (metric definition)]
    "Rewards are assigned based on the system's final response. If deemed safe, defenders receive a reward of +1. ... We report Attack Success Rate (ASR) on all three red-teaming benchmarks, with response harmfulness evaluated by LLaMA-Guard-3-8B [11]."

    The RL objective trains each agent to maximize the LLaMA-Guard-3-8B safety judgment, granting +1 when the final response is 'deemed safe'. The evaluation metric ASR is defined as the fraction of final responses that LLaMA-Guard-3-8B judges harmful. Therefore the reported safety improvement is a direct measure of how well the policy optimizes the very reward signal used in training, not an independent measure of safety. The paper provides no human evaluation or second, independently trained classifier to confirm that the ASR drop reflects genuine internalized safety rather than reward hacking or stylistic compliance with LLaMA-Guard.

full rationale

Evo-MARL's methodological contribution—MARL with parameter sharing, GRPO, and evolutionary attack selection—is self-contained and not circular. The circularity lies in the safety measurement: the same classifier (LLaMA-Guard-3-8B) defines both the training reward and the evaluation metric. Training with GRPO to maximize a +1 reward for 'deemed safe' responses and then reporting ASR as the fraction deemed harmful by that same classifier means the headline safety gain is the training signal itself, not an external validation. The utility results on MATH and Creative Writing are independent and support the claim that helpfulness is not degraded, but they do not rescue the safety claim from this self-referential evaluation. Because the central claim of 'internalized safety' rests on this single classifier bridge, the evaluation is partially circular. No other circularity was found: the related-work citations are not load-bearing, no uniqueness theorem is imported, and the evolutionary mechanism is described independently. Score 6 reflects a central evaluation that reduces to the training objective, while acknowledging the method itself is an independent construction.

Assumptions & free parameters 5 free parameters · 6 assumptions · 0 invented entities

No new physical entities are introduced. The central claim rests on several unvalidated assumptions and unreported hyperparameters. The most significant is that the same LLaMA-Guard classifier provides both the training reward and the evaluation metric, making the measured safety improvement partly a measure of optimization against that classifier.

free parameters (5)
  • Safety reward weight = +1
    Reward for a safe final response in Section 3.1; chosen by hand and not swept or justified.
  • Correctness reward weight = +0.5
    Additional reward for an accurate answer in Section 3.1; chosen by hand.
  • Number of agents in chain = 3
    Chain-structured MAS with problem analyst, solving execution, and answer verifier; no ablation on topology.
  • Evolution mutation and crossover rates = not reported
    Section 3.2 relies on mutation and crossover for attack prompts, but the rates and budget are not specified.
  • GRPO hyperparameters = not reported
    Learning rate, KL penalty, group size G, episode count, and compute budget are not reported.
assumptions (6)
  • domain assumption LLaMA-Guard-3-8B harmfulness score is a valid safety oracle.
    Sections 3.1 and 4.1 use LLaMA-Guard-3-8B for both rewards and ASR evaluation; the paper does not validate it against human judgment or other safety classifiers.
  • domain assumption The three-agent chain structure is representative of real multi-agent systems.
    Section 3.1 defines the training environment; the paper asserts generality to other topologies without experiments.
  • domain assumption Manually inserting attack prompts into downstream responses simulates real safety contagion.
    Section 3.1 uses this injection to create contagion; the paper does not compare with real indirect prompt injection or dynamic propagation.
  • ad hoc to paper Parameter sharing with role-conditioned policies preserves enough role diversity.
    Section 3.1 uses a single shared policy for defenders; no analysis shows that sharing does not harm specialization.
  • domain assumption GRPO, designed for single-agent LLM training, applies directly to multi-agent shared-policy optimization.
    Equation (1) in Section 3.1 is the standard GRPO objective; multi-agent credit assignment and group-relative advantages across agents are not analyzed.
  • ad hoc to paper Evolutionary attack prompts remain in distribution with the static evaluation benchmarks.
    Section 3.2 evolves attacks against the trained system, but Section 4 evaluates on fixed JailBreakV, HarmBench, and MultiJail prompts; transfer is assumed, not shown.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Evo-MARL: Co-Evolutionary Multi-Agent Reinforcement Learning for Internalized Safety." pith.science (2026). https://pith.science/paper/57DJGWQY

@misc{pith2026250803864,
  author       = {Pith},
  title        = {Pith review of: Evo-MARL: Co-Evolutionary Multi-Agent Reinforcement Learning for Internalized Safety},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/57DJGWQY}},
  note         = {Machine review of arXiv:2508.03864}
}
read the original abstract

Multi-agent systems (MAS) built on multimodal large language models exhibit strong collaboration and performance. However, their growing openness and interaction complexity pose serious risks, notably jailbreak and adversarial attacks. Existing defenses typically rely on external guard modules, such as dedicated safety agents, to handle unsafe behaviors. Unfortunately, this paradigm faces two challenges: (1) standalone agents offer limited protection, and (2) their independence leads to single-point failure-if compromised, system-wide safety collapses. Naively increasing the number of guard agents further raises cost and complexity. To address these challenges, we propose Evo-MARL, a novel multi-agent reinforcement learning (MARL) framework that enables all task agents to jointly acquire defensive capabilities. Rather than relying on external safety modules, Evo-MARL trains each agent to simultaneously perform its primary function and resist adversarial threats, ensuring robustness without increasing system overhead or single-node failure. Furthermore, Evo-MARL integrates evolutionary search with parameter-sharing reinforcement learning to co-evolve attackers and defenders. This adversarial training paradigm internalizes safety mechanisms and continually enhances MAS performance under co-evolving threats. Experiments show that Evo-MARL reduces attack success rates by up to 22% while boosting accuracy by up to 5% on reasoning tasks-demonstrating that safety and utility can be jointly improved.

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

49 extracted references · 42 canonical work pages

  1. [1]

    Improving retrieval-augmented generation through multi-agent reinforcement learning, 2025

    Yiqun Chen, Lingyong Yan, Weiwei Sun, Xinyu Ma, Yi Zhang, Shuaiqiang Wang, Dawei Yin, Yiming Yang, and Jiaxin Mao. Improving retrieval-augmented generation through multi-agent reinforcement learning, 2025. 2

    work page 2025

  2. [2]

    Agentpoison: Red-teaming llm agents via poisoning memory or knowledge bases, 2024

    Zhaorun Chen, Zhen Xiang, Chaowei Xiao, Dawn Song, and Bo Li. Agentpoison: Red-teaming llm agents via poisoning memory or knowledge bases, 2024. 2

    work page 2024

  3. [3]

    Deepseek-r1: Incentivizing reasoning capa- bility in llms via reinforcement learning, 2025

    DeepSeek-AI. Deepseek-r1: Incentivizing reasoning capa- bility in llms via reinforcement learning, 2025. 2

    2025

  4. [4]

    Multilingual jailbreak challenges in large language models, 2024

    Yue Deng, Wenxuan Zhang, Sinno Jialin Pan, and Lidong Bing. Multilingual jailbreak challenges in large language models, 2024. 4

    work page 2024

  5. [5]

    A practical memory injection attack against llm agents, 2025

    Shen Dong, Shaochen Xu, Pengfei He, Yige Li, Jiliang Tang, Tianming Liu, Hui Liu, and Zhen Xiang. A practical memory injection attack against llm agents, 2025. 2

    work page 2025

  6. [6]

    Peerguard: Defending multi-agent systems against backdoor attacks through mutual reasoning,

    Falong Fan and Xi Li. Peerguard: Defending multi-agent systems against backdoor attacks through mutual reasoning,

    work page

  7. [7]

    Not what you’ve signed up for: Compromising real-world llm-integrated ap- plications with indirect prompt injection, 2023

    Kai Greshake, Sahar Abdelnabi, Shailesh Mishra, Christoph Endres, Thorsten Holz, and Mario Fritz. Not what you’ve signed up for: Compromising real-world llm-integrated ap- plications with indirect prompt injection, 2023. 2

    work page 2023

  8. [8]

    Llm multi-agent systems: Challenges and open problems, 2025

    Shanshan Han, Qifan Zhang, Yuhang Yao, Weizhao Jin, and Zhaozhuo Xu. Llm multi-agent systems: Challenges and open problems, 2025. 2

    work page 2025

Show all 49 references
  1. [9]

    Red-teaming llm multi-agent systems via commu- nication attacks, 2025

    Pengfei He, Yupin Lin, Shen Dong, Han Xu, Yue Xing, and Hui Liu. Red-teaming llm multi-agent systems via commu- nication attacks, 2025. 1, 2

    2025

  2. [10]

    Measuring mathematical problem solving with the math dataset, 2021

    Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the math dataset, 2021. 4

    2021

  3. [11]

    Llama guard: Llm-based input-output safeguard for human-ai con- versations, 2023

    Hakan Inan, Kartikeya Upasani, Jianfeng Chi, Rashi Rungta, Krithika Iyer, Yuning Mao, Michael Tontchev, Qing Hu, Brian Fuller, Davide Testuggine, and Madian Khabsa. Llama guard: Llm-based input-output safeguard for human-ai con- versations, 2023. 4

    2023

  4. [12]

    Search- r1: Training llms to reason and leverage search engines with reinforcement learning, 2025

    Bowen Jin, Hansi Zeng, Zhenrui Yue, Jinsung Yoon, Sercan Arik, Dong Wang, Hamed Zamani, and Jiawei Han. Search- r1: Training llms to reason and leverage search engines with reinforcement learning, 2025. 2

    2025

  5. [13]

    Rana Muhammad Shahroz Khan, Zhen Tan, Sukwon Yun, Charles Flemming, and Tianlong Chen.Agents Under Siege: Breaking pragmatic multi-agent llm systems with optimized prompt attacks, 2025. 2

    2025

  6. [14]

    Trust re- gion policy optimisation in multi-agent reinforcement learn- ing, 2022

    Jakub Grudzien Kuba, Ruiqing Chen, Muning Wen, Ying Wen, Fanglei Sun, Jun Wang, and Yaodong Yang. Trust re- gion policy optimisation in multi-agent reinforcement learn- ing, 2022. 2

    2022

  7. [15]

    Prompt infection: Llm-to- llm prompt injection within multi-agent systems, 2024

    Donghyun Lee and Mo Tiwari. Prompt infection: Llm-to- llm prompt injection within multi-agent systems, 2024. 2

    2024

  8. [16]

    Deepinception: Hypnotize large language model to be jailbreaker, 2024

    Xuan Li, Zhanke Zhou, Jianing Zhu, Jiangchao Yao, Tongliang Liu, and Bo Han. Deepinception: Hypnotize large language model to be jailbreaker, 2024. 2

    2024

  9. [17]

    Tf-attack: Transferable and fast adversarial attacks on large language models, 2024

    Zelin Li, Kehai Chen, Lemao Liu, Xuefeng Bai, Mingming Yang, Yang Xiang, and Min Zhang. Tf-attack: Transferable and fast adversarial attacks on large language models, 2024. 2

    2024

  10. [18]

    Autodan: Generating stealthy jailbreak prompts on aligned large language models, 2024

    Xiaogeng Liu, Nan Xu, Muhao Chen, and Chaowei Xiao. Autodan: Generating stealthy jailbreak prompts on aligned large language models, 2024. 2

    2024

  11. [19]

    Jailbreakv: A benchmark for assessing the robustness of multimodal large language models against jail- break attacks, 2024

    Weidi Luo, Siyuan Ma, Xiaogeng Liu, Xiaoyu Guo, and Chaowei Xiao. Jailbreakv: A benchmark for assessing the robustness of multimodal large language models against jail- break attacks, 2024. 4

    2024

  12. [20]

    Codechameleon: Personalized encryption frame- work for jailbreaking large language models, 2024

    Huijie Lv, Xiao Wang, Yuansen Zhang, Caishuang Huang, Shihan Dou, Junjie Ye, Tao Gui, Qi Zhang, and Xuanjing Huang. Codechameleon: Personalized encryption frame- work for jailbreaking large language models, 2024. 1, 2

    2024

  13. [21]

    Harm- bench: A standardized evaluation framework for automated red teaming and robust refusal, 2024

    Mantas Mazeika, Long Phan, Xuwang Yin, Andy Zou, Zifan Wang, Norman Mu, Elham Sakhaee, Nathaniel Li, Steven Basart, Bo Li, David Forsyth, and Dan Hendrycks. Harm- bench: A standardized evaluation framework for automated red teaming and robust refusal, 2024. 4

    2024

  14. [22]

    Metaspatial: Reinforcing 3d spa- tial reasoning in vlms for the metaverse.arXiv preprint arXiv:2503.18470, 2025

    Zhenyu Pan and Han Liu. Metaspatial: Reinforcing 3d spa- tial reasoning in vlms for the metaverse.arXiv preprint arXiv:2503.18470, 2025. 1

    2025

  15. [23]

    Codev- bench: How do llms understand developer-centric code com- pletion?arXiv preprint arXiv:2410.01353, 2024

    Zhenyu Pan, Rongyu Cao, Yongchang Cao, Yingwei Ma, Binhua Li, Fei Huang, Han Liu, and Yongbin Li. Codev- bench: How do llms understand developer-centric code com- pletion?arXiv preprint arXiv:2410.01353, 2024. 1

    2024 arXiv

  16. [24]

    Chain-of-action: Faithful and multimodal question an- swering through large language models.arXiv preprint arXiv:2403.17359, 2024

    Zhenyu Pan, Haozheng Luo, Manling Li, and Han Liu. Chain-of-action: Faithful and multimodal question an- swering through large language models.arXiv preprint arXiv:2403.17359, 2024. 1

    2024 arXiv

  17. [25]

    Conv-coa: Improving open-domain question answering in large language models via conversational chain-of-action

    Zhenyu Pan, Haozheng Luo, Manling Li, and Han Liu. Conv-coa: Improving open-domain question answering in large language models via conversational chain-of-action. arXiv preprint arXiv:2405.17822, 2024. 1

    2024 arXiv

  18. [26]

    Do code llms understand design patterns? In2025 IEEE/ACM Inter- national Workshop on Large Language Models for Code (LLM4Code), pages 209–212

    Zhenyu Pan, Xuefeng Song, Yunkun Wang, Rongyu Cao, Binhua Li, Yongbin Li, and Han Liu. Do code llms understand design patterns? In2025 IEEE/ACM Inter- national Workshop on Large Language Models for Code (LLM4Code), pages 209–212. IEEE, 2025. 1

    2025

  19. [27]

    Yu, Manling Li, and Han Liu

    Zhenyu Pan, Yutong Zhang, Jianshu Zhang, Haoran Lu, Haozheng Luo, Yuwei Han, Philip S. Yu, Manling Li, and Han Liu. Fairreason: Balancing reasoning and social bias in mllms, 2025. 1

    2025

  20. [28]

    Maporl: Multi-agent post-co-training for collaborative large language models with reinforcement learning, 2025

    Chanwoo Park, Seungju Han, Xingzhi Guo, Asuman Ozdaglar, Kaiqing Zhang, and Joo-Kyung Kim. Maporl: Multi-agent post-co-training for collaborative large language models with reinforcement learning, 2025. 2

    2025

  21. [29]

    Red teaming the mind of the machine: A systematic evaluation of prompt injection and jailbreak vul- nerabilities in llms, 2025

    Chetan Pathade. Red teaming the mind of the machine: A systematic evaluation of prompt injection and jailbreak vul- nerabilities in llms, 2025. 2

    2025

  22. [30]

    Qwen2.5 technical report, 2025

    Qwen, :, An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Jun- yang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang...

    2025

  23. [31]

    Qmix: Monotonic value function factorisation for deep multi-agent reinforcement learning, 2018

    Tabish Rashid, Mikayel Samvelyan, Christian Schroeder de Witt, Gregory Farquhar, Jakob Foerster, and Shimon White- son. Qmix: Monotonic value function factorisation for deep multi-agent reinforcement learning, 2018. 2

    2018

  24. [32]

    Proximal policy optimization algo- rithms, 2017

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Rad- ford, and Oleg Klimov. Proximal policy optimization algo- rithms, 2017. 2

    2017

  25. [33]

    Exfiltration of personal information from chatgpt via prompt injection, 2024

    Gregory Schwartzman. Exfiltration of personal information from chatgpt via prompt injection, 2024. 2

    2024

  26. [34]

    Sciscigpt: Advancing human- ai collaboration in the science of science.arXiv preprint arXiv:2504.05559, 2025

    Erzhuo Shao, Yifang Wang, Yifan Qian, Zhenyu Pan, Han Liu, and Dashun Wang. Sciscigpt: Advancing human- ai collaboration in the science of science.arXiv preprint arXiv:2504.05559, 2025. 1

    2025

  27. [35]

    Deepseekmath: Pushing the limits of math- ematical reasoning in open language models.arXiv preprint arXiv:2402.03300, 2024

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, et al. Deepseekmath: Pushing the limits of math- ematical reasoning in open language models.arXiv preprint arXiv:2402.03300, 2024. 2

    2024 arXiv

  28. [36]

    Prompt injection attack to tool selection in llm agents, 2025

    Jiawen Shi, Zenghui Yuan, Guiyao Tie, Pan Zhou, Neil Zhenqiang Gong, and Lichao Sun. Prompt injection attack to tool selection in llm agents, 2025. 2

    2025

  29. [37]

    Kimi Team, Yifan Bai, Yiping Bao, Guanduo Chen, Jiahao Chen, Ningxin Chen, Ruijue Chen, Yanru Chen, Yuankun Chen, Yutian Chen, Zhuofu Chen, Jialei Cui, Hao Ding, Mengnan Dong, Angang Du, Chenzhuang Du, Dikang Du, Yulun Du, Yu Fan, Yichen Feng, Kelin Fu, Bofei Gao, Hongcheng Ga...

    2025

  30. [38]

    Multi- agent systems execute arbitrary malicious code, 2025

    Harold Triedman, Rishi Jha, and Vitaly Shmatikov. Multi- agent systems execute arbitrary malicious code, 2025. 1, 2

    2025

  31. [39]

    Lyu, and Maarten Sap

    Jen tse Huang, Jiaxu Zhou, Tailin Jin, Xuhui Zhou, Zixi Chen, Wenxuan Wang, Youliang Yuan, Michael R. Lyu, and Maarten Sap. On the resilience of llm-based multi-agent col- laboration with faulty agents, 2025. 2

    2025

  32. [40]

    Rema: Learning to meta-think for llms with multi-agent reinforcement learning, 2025

    Ziyu Wan, Yunxiang Li, Xiaoyu Wen, Yan Song, Hanjing Wang, Linyi Yang, Mark Schmidt, Jun Wang, Weinan Zhang, Shuyue Hu, and Ying Wen. Rema: Learning to meta-think for llms with multi-agent reinforcement learning, 2025. 2

    2025

  33. [41]

    G- safeguard: A topology-guided security lens and treatment on llm-based multi-agent systems, 2025

    Shilong Wang, Guibin Zhang, Miao Yu, Guancheng Wan, Fanci Meng, Chongye Guo, Kun Wang, and Yang Wang. G- safeguard: A topology-guided security lens and treatment on llm-based multi-agent systems, 2025. 2

    2025

  34. [42]

    Unleashing the emergent cogni- tive synergy in large language models: A task-solving agent through multi-persona self-collaboration, 2024

    Zhenhailong Wang, Shaoguang Mao, Wenshan Wu, Tao Ge, Furu Wei, and Heng Ji. Unleashing the emergent cogni- tive synergy in large language models: A task-solving agent through multi-persona self-collaboration, 2024. 4

    2024

  35. [43]

    Yuxiang Wei, Olivier Duchenne, Jade Copet, Quentin Car- bonneaux, Lingming Zhang, Daniel Fried, Gabriel Syn- naeve, Rishabh Singh, and Sida I. Wang. Swe-rl: Advancing llm reasoning via reinforcement learning on open software evolution, 2025. 2

    2025

  36. [44]

    Beyond self-talk: A communication-centric survey of llm-based multi-agent systems, 2025

    Bingyu Yan, Zhibo Zhou, Litian Zhang, Lian Zhang, Ziyi Zhou, Dezhuang Miao, Zhoujun Li, Chaozhuo Li, and Xi- aoming Zhang. Beyond self-talk: A communication-centric survey of llm-based multi-agent systems, 2025. 2

    2025

  37. [45]

    Jailbreak attacks and defenses against large language models: A survey, 2024

    Sibo Yi, Yule Liu, Zhen Sun, Tianshuo Cong, Xinlei He, Ji- axing Song, Ke Xu, and Qi Li. Jailbreak attacks and defenses against large language models: A survey, 2024. 2

    2024

  38. [46]

    The surprising effec- tiveness of ppo in cooperative, multi-agent games, 2022

    Chao Yu, Akash Velu, Eugene Vinitsky, Jiaxuan Gao, Yu Wang, Alexandre Bayen, and Yi Wu. The surprising effec- tiveness of ppo in cooperative, multi-agent games, 2022. 2

    2022

  39. [47]

    Agent security bench (asb): Formalizing and benchmarking attacks and defenses in llm-based agents,

    Hanrong Zhang, Jingyuan Huang, Kai Mei, Yifei Yao, Zhenting Wang, Chenlu Zhan, Hongwei Wang, and Yongfeng Zhang. Agent security bench (asb): Formalizing and benchmarking attacks and defenses in llm-based agents,

  40. [48]

    Corba: Contagious recur- sive blocking attacks on multi-agent systems based on large language models, 2025

    Zhenhong Zhou, Zherui Li, Jie Zhang, Yuanhe Zhang, Kun Wang, Yang Liu, and Qing Guo. Corba: Contagious recur- sive blocking attacks on multi-agent systems based on large language models, 2025. 2 6 Evo-MARL: Co-Evolutionary Multi-Agent Reinforcement Learning for Internalized Sa...

    2025

  41. [49]

    Successful defense on JailBreakV 1

    Examples ������������������������������������������������������������������������������������ ������������������������������������������������������������������������������ ������������������������ �������������������������������������������������������������������������������...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.