AutoRAS: Learning Robust Agentic Systems with Primitive Representations

Congyu Guo; Guoshun Nan; Hua Wang; Jingfeng Zhang; Jingru Shan; Ji Zhang; Xuancheng Zhu; Yang Yue; Yuyang Ma; Zihan Dou

arxiv: 2606.21445 · v1 · pith:6PK7YWSAnew · submitted 2026-06-19 · 💻 cs.AI

AutoRAS: Learning Robust Agentic Systems with Primitive Representations

Yang Yue , Xuancheng Zhu , Yuyang Ma , Guoshun Nan , Zihan Dou , Jingru Shan , Congyu Guo , Ji Zhang

show 2 more authors

Hua Wang Jingfeng Zhang

This is my paper

Pith reviewed 2026-06-26 14:18 UTC · model grok-4.3

classification 💻 cs.AI

keywords robust agentic systemssymbolic primitivesautomated designLLM agentsadversarial robustnessmulti-agent workflowssequence optimization

0 comments

The pith

AutoRAS designs robust agentic systems by optimizing sequences of symbolic primitives with execution-derived safety signals.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces AutoRAS to automate the design of agentic systems built from large language models, shifting focus from performance alone to robustness against external attacks and internal failures. It represents systems as sequences of symbolic primitives that capture both how agents connect and what actions they take. These sequences are optimized directly using safety signals from executions and flow-based objectives at the sequence level. Experiments indicate that the resulting systems outperform prior handcrafted or generated workflows in standard tasks and under adversarial conditions, while showing the least drop in performance when attacked. A sympathetic reader would care because scaling multi-agent LLM setups has so far left them brittle, and an automated method that bakes in robustness could change how reliable such systems become in practice.

Core claim

AutoRAS formulates system design as generating a sequence of symbolic primitives that jointly encode structural connectivity and behavioral actions, and learns to optimize this sequence using execution-derived safety signals and flow-based sequence-level objectives. Extensive experiments show that AutoRAS achieves the best performance in both vanilla and adversarial settings, with the smallest performance degradation under attacks. Further analyses demonstrate strong transferability, stable optimization behavior, stability across primitive sets, and favorable cost trade-offs.

What carries the argument

Sequences of symbolic primitives that jointly encode structural connectivity and behavioral actions, optimized using execution-derived safety signals and flow-based sequence-level objectives.

If this is right

AutoRAS produces agentic systems that maintain higher performance than prior methods when facing adversarial inputs.
The smallest degradation under attacks follows directly from the safety-signal and flow-based optimization.
The approach transfers across tasks while keeping optimization stable.
Performance remains consistent when the set of available primitives changes.
The method achieves these gains at favorable computational cost compared with alternatives.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

Similar primitive-sequence optimization could be applied to single-agent LLM pipelines to improve their robustness without manual redesign.
The symbolic representation may make it easier to audit or edit agent behaviors after optimization.
If the safety signals can be computed cheaply in new domains, the method might extend to non-LLM agentic systems such as robotic task planners.

Load-bearing premise

Optimizing sequences of symbolic primitives with execution-derived safety signals and flow-based sequence-level objectives will reliably produce agentic systems that are robust to external adversaries and internal failures.

What would settle it

A direct comparison on a held-out adversarial benchmark where AutoRAS exhibits equal or greater performance degradation under attacks than the strongest baseline methods.

Figures

Figures reproduced from arXiv: 2606.21445 by Congyu Guo, Guoshun Nan, Hua Wang, Jingfeng Zhang, Jingru Shan, Ji Zhang, Xuancheng Zhu, Yang Yue, Yuyang Ma, Zihan Dou.

**Figure 1.** Figure 1: Challenges in designing robust agentic systems. i): system design must jointly specify topology and behaviors while preserving flexibility. ii) agentic systems face diverse and evolving risks, whose specific failure modes are difficult to anticipate in advance. iii) different agentic system designs may exhibit similar performance, creating ambiguous learning signals and complicating optimization. sive and … view at source ↗

**Figure 2.** Figure 2: Overview of AutoRAS. We begin with a repository of primitives. Given a query with earlier safety signal s, the system generates a primitive sequence under the forward policy, then compiled into an executable workflow. The workflow is executed with detailed logging, and the monitor inspects traces. Both numeric rewards and textual feedback are then fed back into optimization: trajectory balance shapes the p… view at source ↗

**Figure 3.** Figure 3: Hyperparameter sensitivity analysis of AUTORAS. Obs.❺ Hyperparameter trends reveal diminishing returns in capacity and sampling, and clear robustness–utility tradeoffs. Structural parameters exhibit clear saturation: increasing the sequence length beyond L=16 or the sampling count beyond K=4 provides only marginal gains while adding overhead. Similarly, performance saturates with respect to the number o… view at source ↗

**Figure 4.** Figure 4: 8 12 16 20 24 L for MSMARCO 80 82 84 86 88 90 92 94 (a) Vanilla Attack 2 4 6 8 K for MSMARCO 80 82 84 86 88 90 92 94 (b) Vanilla Attack 2 4 6 8 K for ProgramDev 50 55 60 65 70 75 (c) Vanilla Attack [PITH_FULL_IMAGE:figures/full_fig_p026_4.png] view at source ↗

**Figure 5.** Figure 5: Case study and visualization of AUTORAS 0 20 40 60 80 100 120 140 160 Number of Training Trajectories 0 10 20 30 40 50 60 70 80 Loss Raw Loss Smoothed Loss [PITH_FULL_IMAGE:figures/full_fig_p029_5.png] view at source ↗

**Figure 6.** Figure 6: Trajectory Balance loss against the number of training trajectories. 29 [PITH_FULL_IMAGE:figures/full_fig_p029_6.png] view at source ↗

read the original abstract

The automated design of agentic systems offers a promising pathway for scaling large language models (LLMs) beyond single-agent reasoning. While prior work has advanced task performance through handcrafted or automatically generated multi-agent workflows, robustness is often treated as an afterthought, leaving systems vulnerable to external adversaries and internal failures. We propose AutoRAS, a framework for the Automated design of Robust Agentic Systems. AutoRAS formulates system design as generating a sequence of symbolic primitives that jointly encode structural connectivity and behavioral actions, and learns to optimize this sequence using execution-derived safety signals and flow-based sequence-level objectives. Extensive experiments show that AutoRAS achieves the best performance in both vanilla and adversarial settings, with the smallest performance degradation under attacks. Further analyses demonstrate strong transferability, stable optimization behavior, stability across primitive sets, and favorable cost trade-offs. Our code is available at $\href{https://github.com/guohezuy/AutoRAS}{\text{this https URL}}$.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

AutoRAS turns multi-agent LLM design into sequence optimization over symbolic primitives using execution safety signals, with experiments backing better robustness than prior approaches.

read the letter

This paper gives a workable method for automatically designing robust multi-agent LLM systems using symbolic primitives optimized on execution feedback.

It frames the problem as learning sequences of these primitives that encode both the structure and the behaviors of the agents. The optimization uses safety signals from running the systems and flow-based objectives at the sequence level. This is new compared to prior work on generating workflows, because it directly incorporates robustness into the learning process.

The experiments show it outperforming others in normal conditions and under attacks, with the smallest performance drop when things go wrong. They also report good transfer to new tasks, stable training, consistency when changing the primitive library, and reasonable compute costs. Having the code out helps verify and extend the results.

The main soft spot is that everything hinges on those empirical comparisons. The design seems internally consistent, with no obvious circularity or invented metrics. But the strength of the robustness claim depends on whether the adversarial settings match real threats and if the baselines are competitive. From what's described, it holds up.

This work is for people building or studying multi-agent systems with LLMs who want an automated route to reliability instead of hand-designed teams. Anyone looking at practical deployment of agent teams would get value from the method and the results.

I'd recommend sending it for peer review. The combination of a distinct technical approach and supporting experiments makes it worth a referee's time.

Referee Report

0 major / 3 minor

Summary. The manuscript introduces AutoRAS, a framework for automated design of robust agentic systems. Agentic systems are represented as sequences of symbolic primitives that jointly encode structural connectivity and behavioral actions; these sequences are optimized using execution-derived safety signals together with flow-based sequence-level objectives. The central empirical claim is that AutoRAS attains the highest performance in both vanilla and adversarial settings while exhibiting the smallest degradation under attack, together with transferability, stable optimization, stability across primitive sets, and favorable cost trade-offs. Code is released at the cited GitHub repository.

Significance. If the reported gains are reproducible, the work supplies a concrete method for embedding robustness into the automated construction of multi-agent LLM systems rather than treating it as a post-hoc concern. The combination of symbolic primitive representations with execution-derived signals offers a structured, optimizable interface between high-level design and low-level reliability. Public code availability strengthens the contribution by enabling direct verification and follow-on research.

minor comments (3)

Abstract: the phrase 'extensive experiments' would be more informative if it briefly named the primary task domains or benchmark suites used for the vanilla and adversarial evaluations.
Section 3 (method): the flow-based sequence-level objective is introduced without an explicit equation reference in the surrounding text; adding a numbered equation would improve traceability when the optimization procedure is later invoked in the experiments.
Figure 4 and associated caption: axis labels and legend entries are legible but the caption does not state the number of independent runs or the precise attack strength used, which would aid interpretation of the degradation curves.

Simulated Author's Rebuttal

0 responses · 0 unresolved

We thank the referee for their positive summary of AutoRAS, recognition of its significance for embedding robustness into automated multi-agent design, and recommendation of minor revision. We are pleased that the combination of symbolic primitives with execution-derived signals is viewed as a structured contribution, and that code release is noted as strengthening verifiability.

Circularity Check

0 steps flagged

No significant circularity detected

full rationale

The paper presents AutoRAS as an optimization framework that generates sequences of symbolic primitives and tunes them via external execution-derived safety signals plus flow-based objectives. Performance claims rest on empirical comparisons in vanilla and adversarial settings rather than any derivation that reduces to fitted parameters, self-definitions, or self-citation chains. No equations, uniqueness theorems, or ansatzes are shown to be smuggled in or renamed; the reported gains are presented as outcomes of the described procedure against external benchmarks. This is the common case of a self-contained empirical method.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Abstract-only review supplies no information on free parameters, background axioms, or newly postulated entities.

pith-pipeline@v0.9.1-grok · 5718 in / 1014 out tokens · 16160 ms · 2026-06-26T14:18:02.885487+00:00 · methodology

discussion (0)

Reference graph

Works this paper leans on

11 extracted references

[1]

We employ the implementation from (Wei et al., 2022)

CoT.Chain-of-Thought (CoT) prompting guides LLM agents to break down reasoning into sequential steps rather than generating direct answers. We employ the implementation from (Wei et al., 2022). 13 AutoRAS: Learning Robust Agentic Systems with Primitive Representations 2.Self-consistency.To enhance robustness, we aggregate six CoT-generated responses (Wang...

2022
[2]

4.DyLAN.We instantiate six LLM-agents for handling the problem and 1 ranker for evaluating the generated answer set

LLM-Debate.We instantiate six LLM-agents, each assigned a distinct role, which participate in up to two rounds of debate, after which the final decision is determined via majority voting(Du et al., 2023). 4.DyLAN.We instantiate six LLM-agents for handling the problem and 1 ranker for evaluating the generated answer set. (Liu et al., 2023b). 5.G-Safeguard....

2023
[3]

do A then B

AFlow.AFlow(Zhang et al., 2025e) operates with bothGPT-4O-MINIandCLAUDE-3.5-SONNET. To maintain fairness under homogeneous conditions, we restrict AFlow toGPT-4O-MINIand setMAX ITERATION=20. 10.MaAS.We follow the official configuration specified in (Zhang et al., 2025a) A.3. Attack Details We adopt a unified attack protocol to evaluate single-agent baseli...
[4]

functional_block: Domain-specific guidance based on the query topic (max 2 sentences, 100 chars)
[5]

What is the acceleration due to gravity on Mars?

safety_block: Safety guidance based on the provided safety signals (max 2 sentences, 100 chars) Examples: Query: "What is the acceleration due to gravity on Mars?" Safety Signals: [] functional_block: As a physicist: Start with fundamental principles and show clear unit conversions. safety_block: Verify input completeness and check calculation accuracy. Q...
[9]

After your complete analysis, on the LAST LINE, write only the letter of your answer (A, B, C, or D)

Select the best answer Your reply must be less than 100 words of reasoning. After your complete analysis, on the LAST LINE, write only the letter of your answer (A, B, C, or D). Additional guidance: As a chemistry operator: Refer to solubility rules for precise examples and exceptions. OUTPUT: Step 1: The question asks about solubility rules for various c...
[13]

After your complete analysis, on the LAST LINE, write only the letter of your answer (A, B, C, or D)

Select the best answer Your reply must be less than 100 words of reasoning. After your complete analysis, on the LAST LINE, write only the letter of your answer (A, B, C, or D). Additional guidance: As a chemistry operator: Refer to solubility rules for precise examples and exceptions. OUTPUT: Analysis: A is incorrect because not all chlorides, bromides, ...
[14]

Understand what the question is asking
[15]

Evaluate each option systematically
[16]

Eliminate incorrect options with reasoning
[17]

After your complete analysis, on the LAST LINE, write only the letter of your answer (A, B, C, or D)

Select the best answer Your reply must be less than 100 words of reasoning. After your complete analysis, on the LAST LINE, write only the letter of your answer (A, B, C, or D). Additional guidance: As a chemistry operator: Refer to solubility rules for precise examples and exceptions. OUTPUT: Let’s analyze each option based on solubility rules: A: Not al...

arXiv 2025

[1] [1]

We employ the implementation from (Wei et al., 2022)

CoT.Chain-of-Thought (CoT) prompting guides LLM agents to break down reasoning into sequential steps rather than generating direct answers. We employ the implementation from (Wei et al., 2022). 13 AutoRAS: Learning Robust Agentic Systems with Primitive Representations 2.Self-consistency.To enhance robustness, we aggregate six CoT-generated responses (Wang...

2022

[2] [2]

4.DyLAN.We instantiate six LLM-agents for handling the problem and 1 ranker for evaluating the generated answer set

LLM-Debate.We instantiate six LLM-agents, each assigned a distinct role, which participate in up to two rounds of debate, after which the final decision is determined via majority voting(Du et al., 2023). 4.DyLAN.We instantiate six LLM-agents for handling the problem and 1 ranker for evaluating the generated answer set. (Liu et al., 2023b). 5.G-Safeguard....

2023

[3] [3]

do A then B

AFlow.AFlow(Zhang et al., 2025e) operates with bothGPT-4O-MINIandCLAUDE-3.5-SONNET. To maintain fairness under homogeneous conditions, we restrict AFlow toGPT-4O-MINIand setMAX ITERATION=20. 10.MaAS.We follow the official configuration specified in (Zhang et al., 2025a) A.3. Attack Details We adopt a unified attack protocol to evaluate single-agent baseli...

[4] [4]

functional_block: Domain-specific guidance based on the query topic (max 2 sentences, 100 chars)

[5] [5]

What is the acceleration due to gravity on Mars?

safety_block: Safety guidance based on the provided safety signals (max 2 sentences, 100 chars) Examples: Query: "What is the acceleration due to gravity on Mars?" Safety Signals: [] functional_block: As a physicist: Start with fundamental principles and show clear unit conversions. safety_block: Verify input completeness and check calculation accuracy. Q...

[6] [9]

After your complete analysis, on the LAST LINE, write only the letter of your answer (A, B, C, or D)

Select the best answer Your reply must be less than 100 words of reasoning. After your complete analysis, on the LAST LINE, write only the letter of your answer (A, B, C, or D). Additional guidance: As a chemistry operator: Refer to solubility rules for precise examples and exceptions. OUTPUT: Step 1: The question asks about solubility rules for various c...

[7] [13]

After your complete analysis, on the LAST LINE, write only the letter of your answer (A, B, C, or D)

Select the best answer Your reply must be less than 100 words of reasoning. After your complete analysis, on the LAST LINE, write only the letter of your answer (A, B, C, or D). Additional guidance: As a chemistry operator: Refer to solubility rules for precise examples and exceptions. OUTPUT: Analysis: A is incorrect because not all chlorides, bromides, ...

[8] [14]

Understand what the question is asking

[9] [15]

Evaluate each option systematically

[10] [16]

Eliminate incorrect options with reasoning

[11] [17]

After your complete analysis, on the LAST LINE, write only the letter of your answer (A, B, C, or D)

Select the best answer Your reply must be less than 100 words of reasoning. After your complete analysis, on the LAST LINE, write only the letter of your answer (A, B, C, or D). Additional guidance: As a chemistry operator: Refer to solubility rules for precise examples and exceptions. OUTPUT: Let’s analyze each option based on solubility rules: A: Not al...

arXiv 2025