Pith. sign in

REVIEW 4 major objections 5 minor 79 references

Self-evolving AI agent teams stay safe only when proposal is never promotion: learned changes stay untrusted until held-out evidence, human policy, and explicit authorization allow release.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-14 08:35 UTC pith:IQ67HKBV

load-bearing objection Coherent release-governance contract for self-evolving agents; evaluation cleanly isolates gate rules but mostly measures conformance, not natural proposal quality under imperfect verifiers. the 4 major comments →

arxiv 2607.10878 v1 pith:IQ67HKBV submitted 2026-07-12 cs.AI cs.CL

LOGOS: A Living Logic for AI Agent Teams That Evolve With Humans

Yuma Ichikawa , Yamato Arai , Kosaku Kimura , Akira Sakai , Hiromichi Kobashi This is my paper
classification cs.AI cs.CL
keywords multi-agent systemsself-evolutionrelease governanceagent packshuman oversightverification gatesmisevolutionauditable control loops
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper argues that once AI agents can revise the prompts, memories, skills, tools, roles, and workflows that shape their future behavior, adaptation becomes a software-release problem, not only a capability problem. Logos is a pluggable governance layer that sits on top of existing multi-agent systems: it compiles mixed operational inputs into versioned Agent Packs, runs them through normalized auditable traces, and keeps every learned change isolated. The central rule is simple and strict—proposal is not promotion. A candidate may improve a score or look useful, but it cannot enter the live system until held-out execution evidence, human-owned root policy, and required authorization all pass. That separation lets agents act, ask, learn, and suggest improvements at machine speed while humans keep control of objectives, permissions, approvals, and irreversible side effects without stopping continuous operation.

Core claim

Logos turns multi-agent self-evolution into release engineering and human–agent collaboration into an auditable control loop: every learned prompt, memory, skill, tool, role, or workflow remains an untrusted release candidate until held-out execution evidence, human-controlled policy, and explicit authorization permit its promotion.

What carries the argument

The Agent Pack plus the Promote predicate: a versioned release artifact is promoted only when a held-out paired execution gate, root-policy compatibility, and required authorization all pass, so candidates cannot rewrite evaluators, expand credentials, or mutate live state until that triple succeeds.

Load-bearing premise

The controlled candidate families and generated fault-injection suites are informative enough about real proposal quality, verifier false accepts, and production distribution shift to carry the governance claim.

What would settle it

On naturally occurring agent-proposed updates (not fixed constructed candidate families), measure whether the paired gate still blocks harmful promotions at rates far below ungated or proxy rules; if harmful adoption reappears at comparable rates while the paper’s zero-harmful pattern disappears, the central governance claim fails.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Self-evolving agent teams can keep learning without models approving their own updates.
  • Existing multi-agent stacks can add portable release governance instead of being replaced.
  • Human oversight concentrates on approvals, credentials, holdouts, and irreversible effects while continuous operation continues.
  • Misevolution—adaptation that degrades deployment utility—becomes a measurable release failure via paired held-out gates.
  • Imported skills and cross-deployment knowledge must re-pass target-side gates rather than transfer as automatic trust.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same proposal-is-not-promotion contract could reframe other adaptive AI systems as release pipelines with frozen evaluators outside the learning loop.
  • Production safety claims will need independent natural proposal streams, not only suites generated from the same rules the gate is required to preserve.
  • Verifier quality is the binding constraint: false accepts set how much cascades and promotion gates can actually protect.
  • Structural pack rollback without separate effect controls (previews, transactions, compensation) will keep being confused with undoing real-world side effects.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces LOGOS, a pluggable compile–operate–evolve governance layer for multi-agent systems that keeps learned prompts, memories, skills, tools, roles, and workflows as untrusted release candidates until held-out execution evidence, human-owned root policy, and explicit authorization permit promotion (Eq. 21). It formalizes Agent Packs and genomes, portable traces and effect records, fail-closed verification, a paired-execution adoption gate, and an auditable human–agent control loop, and evaluates these mechanisms with external benchmarks, controlled gate/router/memory studies, generated fault injection, and contract-conformance checks (Q1–Q22).

Significance. If the release-governance contract holds as specified, the paper offers a timely and practically useful framing of agent self-evolution as software release engineering rather than unconstrained self-modification. Strengths include a coherent artifact/event/authority interface (Pack, ζ, χ_eff, R_root), explicit separation of proposal from promotion and of pack rollback from external effects, and unusually careful evidence-class boundaries in §4. The Q3 common-candidate replay is a clean mechanism isolation of gate semantics, and the cascade false-accept accounting (Prop. C.1) is a useful formalization. The work is significant as systems architecture and governance design for persistent agent teams, even if it is not yet a full empirical demonstration of safe self-evolution in the wild.

major comments (4)
  1. [§4.3 Q3 / Table 13] Contribution (3) and the abstract claim that every learned artifact remains untrusted until held-out execution evidence authorizes promotion are only partially supported by the evaluation. Table 13 / Q3 freezes incumbent and candidate outputs and only swaps gate rules over constructed beneficial/neutral/harmful families. That cleanly isolates gate semantics, but does not measure whether natural self-evolution proposals are correctly scored, whether credit assignment selects the right component (§3.6.4), or whether the gate remains reliable under proposal pressure and distribution shift. Either add end-to-end proposal-and-gate studies with naturally generated candidates on held-out operational tasks, or narrow the claim to gate-rule isolation under fixed candidate families.
  2. [§4.9 / Tables 19–23] Many headline safety cells (Tables 19–23; Q9–Q22; also compiler C3–C5 in Table 10 and the safe-delegation suite in Table 19) come from generated grids whose labels and failure modes are produced from the same encoded contract rules LOGOS is required to preserve (§4 intro; Table 7). The paper correctly labels these as conformance/fault-injection evidence, but the abstract and contribution list still present a general governance claim. For load-bearing safety statements (0 unauthorized effects, 0 unsafe imports, 0 leakage), the manuscript should either (i) add external or independently labeled operational cases, or (ii) systematically demote those results to contract-regression evidence and avoid reading them as production safety rates.
  3. [§3.5.3 / Prop. C.1 / Table 20] The promotion predicate (Eq. 21) and cascade analysis make verifier quality first-order: false accepts, coverage failures, and drift are residual risks (§3.5.3; Table 6), and cascade excess error is bounded by reach-conditioned false-accept mass (Prop. C.1; Eqs. 77/83). Table 20 reports a composite false-accept rate of 0.007 on generated release decisions, but this is again a controlled family, not a measurement of gate evidence quality under imperfect scorers on free-form or tool-using tasks. Because held-out execution evidence is the paper’s central promotion signal, the evaluation needs a clearer stress of promotion under deliberately imperfect verifiers (false-accept injection into the paired gate, not only into cascade routing), or an explicit statement that the governance claim is conditional on verifier soundness outside the paper’s measured regimes.
  4. [§4.1–4.2 / Tables 10–12] Q2 (Tables 11–12) shows that compilation can help or hurt depending on suite and model, and the paper correctly treats the compiled team as a candidate behind a validation-route rule. However, the main text still presents multimodal compilation as contribution (1) with strong C3–C5 rates (Table 10) that are generated-template results. The manuscript should more tightly couple the compiler claim to target-side validation evidence and avoid implying that high generated C3–C5 rates establish deployment utility. A short sensitivity or failure analysis of when the compiler’s finite probes pass but end-to-end utility falls (as in the gpt-4o-mini MATH-500 cell) would strengthen the release-engineering story.
minor comments (5)
  1. [§2] Notation density is high early (Π, D, ζ, χ_eff, Pack, G, R_root). A one-page symbol card near the start of §2, or moving more of the formal apparatus to the appendix, would help non-formal readers reach the systems contribution faster.
  2. [§3.6.4 / Appendix G] Several free parameters (δ_min, R_max, T_acc, c_ucb, θ_pass, θ_ask, ε_hi, LORD spending) are acknowledged but not sensitivity-tested. Even a brief appendix sweep on the Q3 gate thresholds would help readers assess robustness of the reported 0% harmful adoption.
  3. [Figures 1–2] Figure 1 and Figure 2 are conceptually clear; ensure that the final PDF keeps the root-policy boundary visually distinct from the evolvable genome, since that separation is central to the argument.
  4. [§5] Related work is broad and useful; a short comparison table of LOGOS vs. DSPy/ADAS/AFlow/MaAS/Darwin–Gödel Machine on the axes of build interface, promotion gate, root policy, and portable traces would make the positioning sharper.
  5. [Throughout] Typos and polish: arXiv id and dates in the header look consistent, but check for residual phrasing such as duplicated “the”/“and” near long formal paragraphs and ensure all Q-labels in Table 8 match the subsection titles exactly.

Circularity Check

1 steps flagged

No load-bearing circular derivation: LOGOS is an engineering contract with properly labeled mechanism/conformance tests; constructed candidate families make some gate numbers true by design of the harness, not by a fitted scientific prediction.

specific steps
  1. fitted input called prediction [§4.3 Q3 Table 13; Table 8 Q3 row; §4 intro / Table 7]
    "In this replay, the family label and oracle outcome are fixed by the candidate-generation rules. The gate receives only the recorded task-paired outputs, proxy signals, and safety metadata used by the stated rule, not a hidden family label. ... the paired gate adopts 0% harmful-family candidates and 100% beneficial-family candidates in 5,000 fixed decisions ... when cases are generated from the same contract rules that Logos must preserve, the row is a conformance test rather than a statistical generalization claim."

    Beneficial/harmful families are constructed with fixed oracle outcomes aligned to the paired-gate criterion (held-out task-paired gain/regression). Once outputs and labels are fixed that way, a correctly implemented paired gate must accept the beneficial family and reject the harmful family by harness design. Reporting 0% harmful / 100% beneficial adoption is therefore a semantic unit check of the gate rule, not an independent prediction about natural self-evolution proposals. The paper labels this boundary, so it is minor and not load-bearing for the architectural claim.

full rationale

LOGOS does not claim a first-principles quantity derived from data. The central object is an explicit release contract: Promote(Pack';Dt) ⇔ B_gate ∧ B_root ∧ B_auth (Eq. 21), with proposal isolated from live state (Eq. 22) and root policy outside ordinary self-evolution (Eq. 24). That is definitional architecture, not a prediction forced by fitting a target. Cascade non-degradation (Prop. C.1) is a conditional lemma under stated verifier/isolation assumptions, not a circular identity. External task suites (GSM8K, MATH-500, LongMemEval, LoCoMo) and the paper’s own residual-risk statements (verifier false accepts, coverage failure, drift; Table 6; §3.5.3) keep the scientific claim open to failure. The only mild circularity-adjacent pattern is evaluation: Q3 common-candidate replay and Q9–Q22/C3–C5 grids are generated from encoded family/contract rules, so perfect gate/conformance cells are partly forced once the harness labels match the gate criterion. The manuscript repeatedly scopes these as mechanism/conformance evidence, not population estimates (Table 7; §4 intro; Table 8 boundary column). Self-citations (e.g. Arai & Ichikawa 2026) are related-work context, not uniqueness theorems that force the LOGOS contract. Score 2 reflects that one minor by-construction evaluation pattern, not a circular central derivation.

Axiom & Free-Parameter Ledger

6 free parameters · 7 axioms · 5 invented entities

The central claim is an engineering contract, not a theorem from physics-like first principles. It rests on release-engineering axioms (proposal ≠ promotion; root policy outside self-evolution; fail-closed verification), many tunable gate/routing/memory thresholds, and invented release objects (Agent Pack, genome, evolution gate, root envelope). Evaluation further depends on generated fault families and verifier-quality assumptions. These are the unpaid inputs a reader must accept to treat the governance claim as operationally meaningful.

free parameters (6)
  • paired-gate gain margin δ_min and regression budget R_max / ε_reg
    Promotion decisions in Eq. (59) depend on pre-declared margins and tolerated regressions; defaults are deployment choices, not derived constants.
  • cascade acceptance threshold T_acc and UCB exploration constant c_ucb
    Entry selection and escalation in Eqs. (38)–(39) are controlled by hand-set thresholds and priors.
  • memory promotion thresholds θ_pass, ε_stab, value-blend λ, and pruning weights
    Whether episodes become validated lessons or durable skills depends on tunable stability/value screens (Eqs. 50–54).
  • ask-policy threshold θ_ask / attention budget G_att and Ask-F1 blend weight ω
    Human interruption behavior is governed by budgeted heuristic thresholds and reward blending (Eqs. 63–65).
  • risk-classification thresholds ε_hi and n_edit
    High/medium/low risk escalation in Eq. (71) depends on score-drop and edit-footprint cutoffs chosen by policy.
  • anytime-valid / LORD alpha-wealth and spending sequence
    Optional statistical gates and online false-adoption accounting depend on chosen error budgets and spending rules (Appendix D).
axioms (7)
  • domain assumption Proposal is not promotion: candidate packs cannot mutate live deployment, root policy, credentials, final holdouts, or audit sinks until Promote passes.
    Governing principle stated in §1 and formalized in Eqs. (21)–(22); load-bearing for the safety claim.
  • domain assumption Human-owned root policy remains outside ordinary self-evolution and fixes objectives, evaluators, permissions, approvals, effects, and audit.
    Eq. (24) and Figure 2; without this, the system can redefine its own release criteria.
  • domain assumption Fail-closed verification: missing, crashed, inapplicable, or unparsable checks reject rather than pass.
    Eq. (47) and §3.5.3; cascade and evolution gates inherit this trust primitive.
  • domain assumption Cascade non-degradation assumes sound early accepts, state isolation, and fresh fallback execution.
    Section 3.5.1 observation and Proposition C.1; accuracy claims depend on verifier soundness or measured false-accept mass.
  • ad hoc to paper Held-out paired execution on the operational harness is sufficient empirical evidence for adjacent-version non-regression under declared budgets.
    Eqs. (19)–(20), (58)–(59); the paper itself notes this does not prove population non-regression under shift or finite-sample error.
  • ad hoc to paper Generated conformance and fault-injection suites are valid for contract composition claims when labels and templates are disclosed.
    §4 evaluation design; used extensively for Q1/Q5/Q7/Q9–Q22 safety and authority claims.
  • standard math Standard bandit, REINFORCE, Beta-Bernoulli, e-process, and LORD machinery apply under stated sampling/dependence assumptions.
    Used for routing, memory value updates, anytime-valid gates, and online accounting; validity is conditional on Appendix D assumptions.
invented entities (5)
  • Agent Pack / Agent Genome no independent evidence
    purpose: Versioned release unit bundling team, tools, verifiers, probes, policies, and evolvable axes.
    Core invented artifact of the paper; independent evidence is only as strong as implementations and evaluations of this schema.
  • Evolution / paired-execution gate with holdout firewall no independent evidence
    purpose: Isolates candidates and accepts only under held-out task-paired evidence plus policy/approval checks.
    Central control object; mechanism studies support gate semantics, but not production proposal distributions.
  • Root policy envelope R_root no independent evidence
    purpose: Human-owned non-evolvable authority boundary for objectives, credentials, evaluators, effects, and audit.
    Necessary invented control surface for the authority claim; residual risk if administrators misconfigure it is acknowledged.
  • Normalized portable execution trace ζ and effect record χ_eff no independent evidence
    purpose: Make backend behavior auditable and prevent confusing pack rollback with undoing external effects.
    Contract objects enabling cross-framework governance; evidence is design plus adapter conformance, not broad third-party adoption.
  • Verifiable human–agent loop engineering / World-scoped residency no independent evidence
    purpose: Budgeted questions, directive synthesis, approvals, and shared workplace lessons under the same release contract.
    Names the bidirectional control loop; evaluated mainly via mechanism and generated safe-delegation suites.

pith-pipeline@v1.1.0-grok45 · 55109 in / 4357 out tokens · 54450 ms · 2026-07-14T08:35:24.455289+00:00 · methodology

0 comments
read the original abstract

AI agents are evolving from answer engines into persistent teams that use tools, delegate work, learn from experience, and modify the artifacts that shape their future behavior. The defining question for deployment is no longer merely what agents can do, but who controls what they are allowed to become. We introduce logos, a pluggable layer for self-evolution and governance that strengthens existing multiagent frameworks rather than replacing them. logos compiles heterogeneous multimodal inputs, including documents, images, audio, tables, databases, APIs, and human instructions into versioned agent packs containing agents, tools, knowledge, tests, permissions, and policies. During operation, it transforms agent activity into portable, auditable event traces and applies fail-closed verification across frameworks and backends. Every learned prompt, memory, skill, tool, role, or workflow remains an untrusted release candidate until held-out execution evidence, human-controlled policy, and explicit authorization permit its promotion. This architecture enables "verifiable human-agent loop engineering": agents can act, ask, learn, and propose improvements, while humans can steer objectives, permissions, approvals, and irreversible actions without interrupting continuous operation. logos provides a living logic for accountable automation. Agents may evolve at machine speed, but only evidence and human authority can close the loop.

Figures

Figures reproduced from arXiv: 2607.10878 by Akira Sakai, Hiromichi Kobashi, Kosaku Kimura, Yamato Arai, Yuma Ichikawa.

Figure 1
Figure 1. Figure 1: The Logos compile–operate–evolve lifecycle. Heterogeneous multimodal inputs and a human objective are compiled into a probe-validated or fail-closed Agent Pack. A pluggable execution layer runs the pack across compatible multi-agent frameworks, while candidate changes remain isolated until a proposer-independent pro￾motion gate combining evidence and authority checks permits promotion. requirements, audit … view at source ↗
Figure 2
Figure 2. Figure 2: Root policy sits outside the evolvable genome: candidates can be copied, gated, rejected, or atomically promoted, but cannot rewrite the evaluator, permissions, or final holdout. 3.1 The Agent Pack and the Agent Genome An Agent Pack is a versioned release representation of a MAS instance. It pairs an initial system Π0 of Eq. (3), materialized as agent definitions, procedural skills, and typed tools, with a… view at source ↗
Figure 3
Figure 3. Figure 3: Compiler pipeline from heterogeneous multimodal sources to a probe-checked or fail-closed Agent Pack, with fallback synthesis and finite validate-and-repair checks. back into backend-native configuration through Eq. (8). This separates the release contract from any one orchestration library. Existing frameworks such as OpenHands and OpenAI Agents illustrate the diversity of execution surfaces that an adapt… view at source ↗
Figure 4
Figure 4. Figure 4: Routing and verification engines. The cascade escalates on verifier rejection; the verifier stack separates hard constraints from acceptance validators. where δa is the point mass on action a, and the tiers {(µℓ, κℓ)} are exactly those emitted by the compiler’s blueprint of Eq. (28). This commits the cheap model to easy queries and reserves the expensive model for harder queries, but it cannot improve from… view at source ↗
Figure 5
Figure 5. Figure 5: Gated memory promotion. Raw episodes remain quarantined by default, validated lessons are pro￾moted only under stable task evidence, and durable skills retain provenance and remain subject to target-side re-verification. H0:p ≥ p ⋆ − ξind to refute it, valid at every stopping time by Ville. The two streams can be reported at separate level α, or at αtot/2 each when a single family-wise αtot is desired. The… view at source ↗
Figure 6
Figure 6. Figure 6: The paired adoption loop: propose from failures, assign credit, evaluate baseline and candidate on held-out execution, then accept only when declared gain, regression, and policy criteria pass. task prompt, answer format, benchmark policy, and question scope take precedence, and conflicting guid￾ance is ignored rather than allowed to become a silent override. Appendix G gives secondary bookkeeping details … view at source ↗
Figure 7
Figure 7. Figure 7: Auditable human–agent control through budgeted questions, directive synthesis, risk gating, approval, and audit. Operator-facing observability and asynchronous control. Logos projects the normalized execution trace into an operator-facing event interface that exposes decision-relevant state, not hidden chain-of￾thought. The observable records include delegation edges, active and completed tasks, tool calls… view at source ↗
Figure 8
Figure 8. Figure 8: Safety-power frontier for the evolution gate ablation. A larger model-independent robustness replay, run after the main table was fixed, keeps the qualita￾tive ordering unchanged. Over 5,000 common-candidate decisions (1,000 seeds × five families), no-gate adoption again accepts all candidates with 60.0% harmful adoption and negative utility (−0.156±0.002 per decision); the proxy rule accepts 40.0% harmful… view at source ↗
Figure 9
Figure 9. Figure 9: Verification-gated cascade summary on matched subsets, with n and paired tests in Appendix G. Execution-graded code is the positive regime; gains over the strong single model are not significant. A separate cascade stress isolates the verifier assumption itself. Over 200 seeds and 2,000 generated tasks per seed, the task generator spans fixed failure-mode templates rather than independent deployment incide… view at source ↗
Figure 10
Figure 10. Figure 10: Collective-router cost–accuracy frontier on fixed held-out replay. replacement for the strong reference. The Q7 accuracy rows are not equal-token or equal-latency compar￾isons. Cascade runs may spend additional calls and verifier executions before accepting an answer. We therefore interpret the result as a routing frontier under the recorded harness, with normalized cost and mean-call summaries reported i… view at source ↗
Figure 11
Figure 11. Figure 11: Grouped view of the official-scale direct-mode sweep [PITH_FULL_IMAGE:figures/full_fig_p055_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Reliability decay on repeated τ 2 and τ 3 trials [PITH_FULL_IMAGE:figures/full_fig_p056_12.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

79 extracted references · 1 canonical work pages · 1 internal anchor

  1. [1]

    New Tools for Building Agents , year =

  2. [2]

    2025 , month = apr, howpublished =

    Announcing the. 2025 , month = apr, howpublished =

    2025

  3. [3]

    2025 , howpublished =

    Peter Belcak and Greg Heinrich and Shizhe Diao and Yonggan Fu and Xin Dong and Saurav Muralidharan and Yingyan Celine Lin and Pavlo Molchanov , title =. 2025 , howpublished =. doi:10.48550/arXiv.2506.02153 , url =

    Pith/arXiv arXiv doi:10.48550/arxiv.2506.02153 2025

  4. [4]

    2025 , month = nov, howpublished =

    John Irwin and Kai Greshake , title =. 2025 , month = nov, howpublished =

    2025

  5. [5]

    2023 , doi =

    Artificial Intelligence Risk Management Framework (. 2023 , doi =

    2023

  6. [6]

    2025 , month = feb, howpublished =

    Agentic. 2025 , month = feb, howpublished =

    2025

  7. [7]

    2026 , howpublished =

    2026

  8. [8]

    doi:10.48550/arXiv.2507.21046 , url =

    A Survey of Self-Evolving Agents: On Path to Artificial Super Intelligence , year =. doi:10.48550/arXiv.2507.21046 , url =

    Pith/arXiv arXiv doi:10.48550/arxiv.2507.21046

  9. [9]

    2026 , howpublished =

    Jenny Zhang and Bingchen Zhao and Wannan Yang and Jakob Foerster and Jeff Clune and Minqi Jiang and Sam Devlin and Tatiana Shavrina , title =. 2026 , howpublished =. doi:10.48550/arXiv.2603.19461 , url =

    doi:10.48550/arxiv.2603.19461 2026

  10. [10]

    2025 , howpublished =

    Shuai Shao and Qihan Ren and Chen Qian and Boyi Wei and Dadi Guo and Jingyi Yang and Xinhao Song and Linfeng Zhang and Weinan Zhang and Dongrui Liu and Jing Shao , title =. 2025 , howpublished =. doi:10.48550/arXiv.2509.26354 , url =

    doi:10.48550/arxiv.2509.26354 2025

  11. [11]

    EVE-Agent: Evidence-Verifiable Self-Evolving Agents

    Yamato Arai and Yuma Ichikawa , title =. 2026 , howpublished =. doi:10.48550/arXiv.2605.22905 , url =

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2605.22905 2026

  12. [12]

    Agrawal and Shangyin Tan and Dilara Soylu and Noah Ziems and Rishi Khare and Krista Opsahl-Ong and Arnav Singhvi and Herumb Shandilya and Michael J

    Lakshya A. Agrawal and Shangyin Tan and Dilara Soylu and Noah Ziems and Rishi Khare and Krista Opsahl-Ong and Arnav Singhvi and Herumb Shandilya and Michael J. Ryan and Meng Jiang and Christopher Potts and Koushik Sen and Alexandros G. Dimakis and Ion Stoica and Dan Klein and Matei Zaharia and Omar Khattab , title =. 2025 , howpublished =. doi:10.48550/ar...

    Pith/arXiv arXiv doi:10.48550/arxiv.2507.19457 2025

  13. [13]

    2025 , howpublished =

    Maxime Robeyns and Martin Szummer and Laurence Aitchison , title =. 2025 , howpublished =. doi:10.48550/arXiv.2504.15228 , url =

    Pith/arXiv arXiv doi:10.48550/arxiv.2504.15228 2025

  14. [14]

    2025 , howpublished =

    Jenny Zhang and Shengran Hu and Cong Lu and Robert Lange and Jeff Clune , title =. 2025 , howpublished =. doi:10.48550/arXiv.2505.22954 , url =

    Pith/arXiv arXiv doi:10.48550/arxiv.2505.22954 2025

  15. [15]

    2024 , howpublished =

    Shengran Hu and Cong Lu and Jeff Clune , title =. 2024 , howpublished =. doi:10.48550/arXiv.2408.08435 , url =

    Pith/arXiv arXiv doi:10.48550/arxiv.2408.08435 2024

  16. [16]

    2024 , howpublished =

    Jiayi Zhang and Jinyu Xiang and Zhaoyang Yu and Fengwei Teng and Xionghui Chen and Jiaqi Chen and Mingchen Zhuge and Xin Cheng and Sirui Hong and Jinlin Wang and Bingnan Zheng and Bang Liu and Yuyu Luo and Chenglin Wu , title =. 2024 , howpublished =. doi:10.48550/arXiv.2410.10762 , url =

    Pith/arXiv arXiv doi:10.48550/arxiv.2410.10762 2024

  17. [17]

    International Conference on Learning Representations , year =

    Yu Shang and Yu Li and Keyu Zhao and Likai Ma and Jiahe Liu and Fengli Xu and Yong Li , title =. International Conference on Learning Representations , year =. 2410.06153 , archiveprefix =

    Pith/arXiv arXiv

  18. [18]

    2025 , howpublished =

    Guibin Zhang and Luyang Niu and Junfeng Fang and Kun Wang and Lei Bai and Xiang Wang , title =. 2025 , howpublished =. doi:10.48550/arXiv.2502.04180 , url =

    Pith/arXiv arXiv doi:10.48550/arxiv.2502.04180 2025

  19. [19]

    White and Doug Burger and Chi Wang , title =

    Qingyun Wu and Gagan Bansal and Jieyu Zhang and Yiran Wu and Beibin Li and Erkang Zhu and Li Jiang and Xiaoyun Zhang and Shaokun Zhang and Jiale Liu and Ahmed Hassan Awadallah and Ryen W. White and Doug Burger and Chi Wang , title =. 2023 , howpublished =. doi:10.48550/arXiv.2308.08155 , url =

    Pith/arXiv arXiv doi:10.48550/arxiv.2308.08155 2023

  20. [20]

    International Conference on Learning Representations , year =

    Sirui Hong and Mingchen Zhuge and Jiaqi Chen and Xiawu Zheng and Yuheng Cheng and Ceyao Zhang and Jinlin Wang and Zili Wang and Steven Ka Shing Yau and Zijuan Lin and Liyang Zhou and Chenyu Ran and Lingfeng Xiao and Chenglin Wu and J. International Conference on Learning Representations , year =. 2308.00352 , archiveprefix =

    Pith/arXiv arXiv

  21. [21]

    Xu and Xiangru Tang and Mingchen Zhuge and Jiayi Pan and Yueqi Song and Bowen Li and Jaskirat Singh and Hoang H

    Xingyao Wang and Boxuan Li and Yufan Song and Frank F. Xu and Xiangru Tang and Mingchen Zhuge and Jiayi Pan and Yueqi Song and Bowen Li and Jaskirat Singh and Hoang H. Tran and Fuqiang Li and Ren Ma and Mingzhang Zheng and Bill Qian and Yanjun Shao and Niklas Muennighoff and Yizhe Zhang and Binyuan Hui and Junyang Lin and Robert Brennan and Hao Peng and H...

    Pith/arXiv arXiv

  22. [22]

    Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics , year =

    Chen Qian and Wei Liu and Hongzhang Liu and Nuo Chen and Yufan Dang and Jiahao Li and Cheng Yang and Weize Chen and Yusheng Su and Xin Cong and Juyuan Xu and Dahai Li and Zhiyuan Liu and Maosong Sun , title =. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics , year =. 2307.07924 , archiveprefix =

    Pith/arXiv arXiv

  23. [23]

    Advances in Neural Information Processing Systems , year =

    Guohao Li and Hasan Abed Al Kader Hammoud and Hani Itani and Dmitrii Khizbullin and Bernard Ghanem , title =. Advances in Neural Information Processing Systems , year =. 2303.17760 , archiveprefix =

    Pith/arXiv arXiv

  24. [24]

    Joshi and Hanna Moazam and Heather Miller and Matei Zaharia and Christopher Potts , title =

    Omar Khattab and Arnav Singhvi and Paridhi Maheshwari and Zhiyuan Zhang and Keshav Santhanam and Sri Vardhamanan and Saiful Haq and Ashutosh Sharma and Thomas T. Joshi and Hanna Moazam and Heather Miller and Matei Zaharia and Christopher Potts , title =. International Conference on Learning Representations , year =. 2310.03714 , archiveprefix =

    Pith/arXiv arXiv

  25. [25]

    Language Agents as Optimizable Graphs , booktitle =

    Mingchen Zhuge and Wenyi Wang and Louis Kirsch and Francesco Faccio and Dmitrii Khizbullin and J. Language Agents as Optimizable Graphs , booktitle =. 2024 , eprint =

    2024

  26. [26]

    2023 , howpublished =

    Lingjiao Chen and Matei Zaharia and James Zou , title =. 2023 , howpublished =. doi:10.48550/arXiv.2305.05176 , url =

    Pith/arXiv arXiv doi:10.48550/arxiv.2305.05176 2023

  27. [27]

    Gonzalez and M

    Isaac Ong and Amjad Almahairi and Vincent Wu and Wei-Lin Chiang and Tianhao Wu and Joseph E. Gonzalez and M. Waleed Kadous and Ion Stoica , title =. 2024 , howpublished =. doi:10.48550/arXiv.2406.18665 , url =

    Pith/arXiv arXiv doi:10.48550/arxiv.2406.18665 2024

  28. [28]

    Gonzalez and Ion Stoica , title =

    Lianmin Zheng and Wei-Lin Chiang and Ying Sheng and Siyuan Zhuang and Zhanghao Wu and Yonghao Zhuang and Zi Lin and Zhuohan Li and Dacheng Li and Eric Xing and Hao Zhang and Joseph E. Gonzalez and Ion Stoica , title =. 2023 , howpublished =. doi:10.48550/arXiv.2306.05685 , url =

    Pith/arXiv arXiv doi:10.48550/arxiv.2306.05685 2023

  29. [29]

    2021 , howpublished =

    Karl Cobbe and Vineet Kosaraju and Mohammad Bavarian and Mark Chen and Heewoo Jun and Lukasz Kaiser and Matthias Plappert and Jerry Tworek and Jacob Hilton and Reiichiro Nakano and Christopher Hesse and John Schulman , title =. 2021 , howpublished =. doi:10.48550/arXiv.2110.14168 , url =

    Pith/arXiv arXiv doi:10.48550/arxiv.2110.14168 2021

  30. [30]

    2024 , howpublished =

    Lunjun Zhang and Arian Hosseini and Hritik Bansal and Mehran Kazemi and Aviral Kumar and Rishabh Agarwal , title =. 2024 , howpublished =. doi:10.48550/arXiv.2408.15240 , url =

    Pith/arXiv arXiv doi:10.48550/arxiv.2408.15240 2024

  31. [31]

    2024 , howpublished =

    Shunyu Yao and Noah Shinn and Pedram Razavi and Karthik Narasimhan , title =. 2024 , howpublished =. doi:10.48550/arXiv.2406.12045 , url =

    Pith/arXiv arXiv doi:10.48550/arxiv.2406.12045 2024

  32. [32]

    Patil and Huanzhi Mao and Fanjia Yan and Charlie Cheng-Jie Ji and Vishnu Suresh and Ion Stoica and Joseph E

    Shishir G. Patil and Huanzhi Mao and Fanjia Yan and Charlie Cheng-Jie Ji and Vishnu Suresh and Ion Stoica and Joseph E. Gonzalez , title =. Proceedings of the 42nd International Conference on Machine Learning , year =

  33. [33]

    2025 , howpublished =

    Victor Barres and Honghua Dong and Soham Ray and Xujie Si and Karthik Narasimhan , title =. 2025 , howpublished =. doi:10.48550/arXiv.2506.07982 , url =

    Pith/arXiv arXiv doi:10.48550/arxiv.2506.07982 2025

  34. [34]

    2026 , howpublished =

    Quan Shi and Alexandra Zytek and Pedram Razavi and Karthik Narasimhan and Victor Barres , title =. 2026 , howpublished =. doi:10.48550/arXiv.2603.04370 , url =

    doi:10.48550/arxiv.2603.04370 2026

  35. [35]

    2021 , howpublished =

    Dan Hendrycks and Collin Burns and Saurav Kadavath and Akul Arora and Steven Basart and Eric Tang and Dawn Song and Jacob Steinhardt , title =. 2021 , howpublished =. doi:10.48550/arXiv.2103.03874 , url =

    Pith/arXiv arXiv doi:10.48550/arxiv.2103.03874 2021

  36. [36]

    Mark Chen and Jerry Tworek and Heewoo Jun and Qiming Yuan and Henrique Ponde de Oliveira Pinto and Jared Kaplan and Harri Edwards and Yuri Burda and Nicholas Joseph and Greg Brockman and Alex Ray and Raul Puri and Gretchen Krueger and Michael Petrov and Heidy Khlaaf and Girish Sastry and Pamela Mishkin and Brooke Chan and Scott Gray and Nick Ryder and Mik...

    Pith/arXiv arXiv doi:10.48550/arxiv.2107.03374 2021

  37. [37]

    2021 , howpublished =

    Jacob Austin and Augustus Odena and Maxwell Nye and Maarten Bosma and Henryk Michalewski and David Dohan and Ellen Jiang and Carrie Cai and Michael Terry and Quoc Le and Charles Sutton , title =. 2021 , howpublished =. doi:10.48550/arXiv.2108.07732 , url =

    Pith/arXiv arXiv doi:10.48550/arxiv.2108.07732 2021

  38. [38]

    Manning , title =

    Zhilin Yang and Peng Qi and Saizheng Zhang and Yoshua Bengio and William Cohen and Ruslan Salakhutdinov and Christopher D. Manning , title =. Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing , year =

    2018

  39. [39]

    Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics , year =

    Dheeru Dua and Yizhong Wang and Pradeep Dasigi and Gabriel Stanovsky and Sameer Singh and Matt Gardner , title =. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics , year =

    2019

  40. [40]

    2024 , howpublished =

    Yujian Gan and Changling Li and Jinxia Xie and Luou Wen and Matthew Purver and Massimo Poesio , title =. 2024 , howpublished =. doi:10.48550/arXiv.2409.06097 , url =

    Pith/arXiv arXiv doi:10.48550/arxiv.2409.06097 2024

  41. [41]

    Rossi and Dinesh Manocha , title =

    Manan Suri and Puneet Mathur and Nedim Lipka and Franck Dernoncourt and Ryan A. Rossi and Dinesh Manocha , title =. 2025 , howpublished =. doi:10.48550/arXiv.2511.08798 , url =

    Pith/arXiv arXiv doi:10.48550/arxiv.2511.08798 2025

  42. [42]

    2026 , howpublished =

    Tu Trinh and Mohamed Elfeki and Guangze Luo and Kelvin Luu and Nathan Hunt and Ernesto Hern. 2026 , howpublished =. doi:10.48550/arXiv.2604.09408 , url =

    Pith/arXiv arXiv doi:10.48550/arxiv.2604.09408 2026

  43. [43]

    2023 , howpublished =

    Minghao Li and Feifan Song and Bowen Yu and Haiyang Yu and Zhoujun Li and Fei Huang and Yongbin Li , title =. 2023 , howpublished =. doi:10.48550/arXiv.2304.08244 , url =

    Pith/arXiv arXiv doi:10.48550/arxiv.2304.08244 2023

  44. [44]

    2024 , howpublished =

    Zhexin Zhang and Shiyao Cui and Yida Lu and Jingzhuo Zhou and Junxiao Yang and Hongning Wang and Minlie Huang , title =. 2024 , howpublished =. doi:10.48550/arXiv.2412.14470 , url =

    Pith/arXiv arXiv doi:10.48550/arxiv.2412.14470 2024

  45. [45]

    Jimenez and John Yang and Alexander Wettig and Shunyu Yao and Kexin Pei and Ofir Press and Karthik Narasimhan , title =

    Carlos E. Jimenez and John Yang and Alexander Wettig and Shunyu Yao and Kexin Pei and Ofir Press and Karthik Narasimhan , title =. International Conference on Learning Representations , year =. 2310.06770 , archiveprefix =

    Pith/arXiv arXiv

  46. [46]

    Advances in Neural Information Processing Systems , year =

    Yubo Wang and Xueguang Ma and Ge Zhang and Yuansheng Ni and Abhranil Chandra and Shiguang Guo and Weiming Ren and Aaran Arulraj and Xuan He and Ziyan Jiang and Tianle Li and Max Ku and Kai Wang and Alex Zhuang and Rongqi Fan and Xiang Yue and Wenhu Chen , title =. Advances in Neural Information Processing Systems , year =. 2406.01574 , archiveprefix =

    Pith/arXiv arXiv

  47. [47]

    International Conference on Learning Representations , year =

    Di Wu and Hongwei Wang and Wenhao Yu and Yuwei Zhang and Kai-Wei Chang and Dong Yu , title =. International Conference on Learning Representations , year =. 2410.10813 , archiveprefix =

    Pith/arXiv arXiv

  48. [48]

    2026 , howpublished =

    Di Wu and Zixiang Ji and Asmi Kawatkar and Bryan Kwan and Jia-Chen Gu and Nanyun Peng and Kai-Wei Chang , title =. 2026 , howpublished =. doi:10.48550/arXiv.2605.12493 , url =

    Pith/arXiv arXiv doi:10.48550/arxiv.2605.12493 2026

  49. [49]

    2024 , howpublished =

    Adyasha Maharana and Dong-Ho Lee and Sergey Tulyakov and Mohit Bansal and Francesco Barbieri and Yuwei Fang , title =. 2024 , howpublished =. doi:10.48550/arXiv.2402.17753 , url =

    Pith/arXiv arXiv doi:10.48550/arxiv.2402.17753 2024

  50. [50]

    Jimenez and Alexander Wettig and Kilian Lieret and Shunyu Yao and Karthik Narasimhan and Ofir Press , title =

    John Yang and Carlos E. Jimenez and Alexander Wettig and Kilian Lieret and Shunyu Yao and Karthik Narasimhan and Ofir Press , title =. 2024 , howpublished =. doi:10.48550/arXiv.2405.15793 , url =

    Pith/arXiv arXiv doi:10.48550/arxiv.2405.15793 2024

  51. [51]

    Advances in Neural Information Processing Systems , year =

    Noah Shinn and Federico Cassano and Edward Berman and Ashwin Gopinath and Karthik Narasimhan and Shunyu Yao , title =. Advances in Neural Information Processing Systems , year =. 2303.11366 , archiveprefix =

    Pith/arXiv arXiv

  52. [52]

    Transactions on Machine Learning Research , year =

    Guanzhi Wang and Yuqi Xie and Yunfan Jiang and Ajay Mandlekar and Chaowei Xiao and Yuke Zhu and Linxi Fan and Anima Anandkumar , title =. Transactions on Machine Learning Research , year =. 2305.16291 , archiveprefix =

    Pith/arXiv arXiv

  53. [53]

    Advances in Neural Information Processing Systems , year =

    Aman Madaan and Niket Tandon and Prakhar Gupta and Skyler Hallinan and Luyu Gao and Sarah Wiegreffe and Uri Alon and Nouha Dziri and Shrimai Prabhumoye and Yiming Yang and Shashank Gupta and Bodhisattwa Prasad Majumder and Katherine Hermann and Sean Welleck and Amir Yazdanbakhsh and Peter Clark , title =. Advances in Neural Information Processing Systems ...

    Pith/arXiv arXiv

  54. [54]

    2024 , howpublished =

    Mert Yuksekgonul and Federico Bianchi and Joseph Boen and Sheng Liu and Pan Lu and Zhi Huang and Carlos Guestrin and James Zou , title =. 2024 , howpublished =. doi:10.48550/arXiv.2406.07496 , url =

    Pith/arXiv arXiv doi:10.48550/arxiv.2406.07496 2024

  55. [55]

    Le and Denny Zhou and Xinyun Chen , title =

    Chengrun Yang and Xuezhi Wang and Yifeng Lu and Hanxiao Liu and Quoc V. Le and Denny Zhou and Xinyun Chen , title =. International Conference on Learning Representations , year =. 2309.03409 , archiveprefix =

    Pith/arXiv arXiv

  56. [56]

    Proceedings of the 41st International Conference on Machine Learning , year =

    Chrisantha Fernando and Dylan Banarse and Henryk Michalewski and Simon Osindero and Tim Rockt. Proceedings of the 41st International Conference on Machine Learning , year =. 2309.16797 , archiveprefix =

    Pith/arXiv arXiv

  57. [57]

    O'Brien and Carrie J

    Joon Sung Park and Joseph C. O'Brien and Carrie J. Cai and Meredith Ringel Morris and Percy Liang and Michael S. Bernstein , title =. Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology , year =. doi:10.1145/3586183.3606763 , url =

    doi:10.1145/3586183.3606763

  58. [58]

    Patil and Ion Stoica and Joseph E

    Charles Packer and Sarah Wooders and Kevin Lin and Vivian Fang and Shishir G. Patil and Ion Stoica and Joseph E. Gonzalez , title =. 2023 , howpublished =. doi:10.48550/arXiv.2310.08560 , url =

    Pith/arXiv arXiv doi:10.48550/arxiv.2310.08560 2023

  59. [59]

    Le and Ed H

    Xuezhi Wang and Jason Wei and Dale Schuurmans and Quoc V. Le and Ed H. Chi and Sharan Narang and Aakanksha Chowdhery and Denny Zhou , title =. International Conference on Learning Representations , year =. 2203.11171 , archiveprefix =

    Pith/arXiv arXiv

  60. [60]

    Finite-Time Analysis of the Multiarmed Bandit Problem , journal =

    Peter Auer and Nicol. Finite-Time Analysis of the Multiarmed Bandit Problem , journal =. 2002 , doi =

    2002

  61. [61]

    Cover , title =

    Thomas M. Cover , title =. Mathematical Finance , volume =. 1991 , doi =

    1991

  62. [62]

    Proceedings of the 47th Annual ACM Symposium on Theory of Computing , pages =

    Cynthia Dwork and Vitaly Feldman and Moritz Hardt and Toniann Pitassi and Omer Reingold and Aaron Roth , title =. Proceedings of the 47th Annual ACM Symposium on Theory of Computing , pages =. 2015 , doi =

    2015

  63. [63]

    Science , volume =

    Cynthia Dwork and Vitaly Feldman and Moritz Hardt and Toniann Pitassi and Omer Reingold and Aaron Roth , title =. Science , volume =. 2015 , doi =

    2015

  64. [64]

    Howard and Aaditya Ramdas and Jon McAuliffe and Jasjeet Sekhon , title =

    Steven R. Howard and Aaditya Ramdas and Jon McAuliffe and Jasjeet Sekhon , title =. The Annals of Statistics , volume =. 2021 , doi =

    2021

  65. [65]

    The Annals of Statistics , volume =

    Adel Javanmard and Andrea Montanari , title =. The Annals of Statistics , volume =. 2018 , doi =

    2018

  66. [66]

    Bandit Algorithms , publisher =

    Tor Lattimore and Csaba Szepesv. Bandit Algorithms , publisher =. 2020 , doi =

    2020

  67. [67]

    2015 , howpublished =

    Jean-Baptiste Mouret and Jeff Clune , title =. 2015 , howpublished =. 1504.04909 , archiveprefix =

    Pith/arXiv arXiv 2015

  68. [68]

    Nemhauser and Laurence A

    George L. Nemhauser and Laurence A. Wolsey and Marshall L. Fisher , title =. Mathematical Programming , volume =. 1978 , doi =

    1978

  69. [69]

    Wainwright and Michael I

    Aaditya Ramdas and Tijana Zrnic and Martin J. Wainwright and Michael I. Jordan , title =. Proceedings of the 21st International Conference on Artificial Intelligence and Statistics , series =. 2018 , url =

    2018

  70. [70]

    Game-Theoretic Statistics and Safe Anytime-Valid Inference , journal =

    Aaditya Ramdas and Peter Gr. Game-Theoretic Statistics and Safe Anytime-Valid Inference , journal =. 2023 , doi =

    2023

  71. [71]

    Journal of the Royal Statistical Society: Series A (Statistics in Society) , volume =

    Glenn Shafer , title =. Journal of the Royal Statistical Society: Series A (Statistics in Society) , volume =. 2021 , doi =

    2021

  72. [72]

    Shapley , title =

    Lloyd S. Shapley , title =. Contributions to the Theory of Games II , editor =. 1953 , doi =

    1953

  73. [73]

    Jean Ville , title =

  74. [74]

    The Annals of Statistics , volume =

    Vladimir Vovk and Ruodu Wang , title =. The Annals of Statistics , volume =. 2021 , doi =

    2021

  75. [75]

    Journal of the Royal Statistical Society Series B: Statistical Methodology , volume =

    Ian Waudby-Smith and Aaditya Ramdas , title =. Journal of the Royal Statistical Society Series B: Statistical Methodology , volume =. 2024 , doi =. 2010.09686 , archiveprefix =

    Pith/arXiv arXiv 2024

  76. [76]

    Williams , title =

    Ronald J. Williams , title =. Machine Learning , volume =. 1992 , doi =

    1992

  77. [77]

    Proceedings of the SIGCHI Conference on Human Factors in Computing Systems , pages =

    Eric Horvitz , title =. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems , pages =. 1999 , doi =

    1999

  78. [78]

    Allen and Curry I

    James E. Allen and Curry I. Guinn and Eric Horvitz , title =. IEEE Intelligent Systems , volume =. 1999 , doi =

    1999

  79. [79]

    Pynadath and Milind Tambe , title =

    Paul Scerri and David V. Pynadath and Milind Tambe , title =. Journal of Artificial Intelligence Research , volume =. 2002 , doi =

    2002