Pith. sign in

REVIEW 2 major objections 4 minor 43 references

Beyond Tier Labels: Role- and Deployment-Dependent Model Substitution in Multi-Call LLM Workflows

T0 review · 2 major / 4 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The value of swapping a medium model for a strong one inside a multi-call LLM workflow is a property of the deployment, not the tier label.

desk verdict A genuinely useful empirical protocol for measuring deployment-conditioned model substitution, but the headline reversal in the abstract is not statistically supported; the conceptual point survives on stronger evidence elsewhere. read the letter →

arxiv 2608.09155 v1 pith:TY7QT26V submitted 2026-08-10 cs.MA

Renxiang Wang , Jiaming Cui This is my paper · ORCID
classification cs.MA
keywords LLMroutingmulti-callworkflowsmodelsubstitutiontierlabelspredicate-actionfactorizationnode-conditionedvalueallocationfootprintcost-qualitytradeoff
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Multi-call LLM workflows distribute a task across many dependent calls, and this paper argues that the benefit of replacing a medium model with a strong one is not a fixed property of the model tier. In solve-merge-verify pipelines of 8–64 calls, the same medium-to-strong substitution lowers frequency-count RMSE from 4.818 to 1.538 in a Mixed Qwen/GPT pool, while in the Qwen-only pool it raises RMSE from 4.843 to 5.091. The paper separates the decision of where to intervene (a risk predicate) from the decision of what model change to apply (a substitution action), and measures the action's value at a specific node. This makes model substitution a deployment-conditioned action: pool, role, scale, and task contract all change whether the action helps. The payoff is a concrete routing sequence—calibrate the action, localize its effect, then optimize placement—so that routers stop inheriting rankings from tier labels.

What carries the argument

The predicate-action factorization is the central object: a risk predicate selects a node context that may warrant intervention (for example, a high-fan-in merge), and a substitution action names the concrete model change to apply there (for example, medium to strong in a particular pool). The associated quantity is node-conditioned substitution value $\Delta_{\mathrm{local}}(z)$, estimated by matched executions on identical inputs under a fixed deployment. EvoCap, a trace-to-constraint allocator, turns observed outcomes and resource use into auditable at-least and at-most tier rules, making every placement attributable. The factorization carries the argument because it makes calibration, role localization, and placement optimization separately testable.

What would settle it

Remove expected_total and observed_total from the VERIFY prompt and re-run the 64-call Mixed condition under failure-aware allocation; if the sparse 10% strong-call operating point with RMSE near 1.5 degrades toward all-medium RMSE, the placement and efficiency claims depend on oracle verification rather than on the deployment-conditioned action value.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that node-conditioned substitution value—the change in loss from applying a named model change at a given workflow node under a fixed deployment—is the right unit of analysis for workflow routing. Across four model ladders, fixed-tier sweeps show that the same 'medium to strong' action has opposite signs: all-strong wins in the Mixed pool (RMSE 1.538 vs 4.818), loses in Qwen-only (RMSE 5.091 vs 4.843), and loses again after a semantic task-and-contract shift (exact match 0.156 vs 0.466 for medium). Input-matched interventions show the effect is role-concentrated: in Mixed, 192 of 192 merge contexts improve while 5,732 of 5,760 solve pairs tie; in Qwen-only, the same merge action improves 126 contexts and harms 66. The conclusion is that tier labels describe an implementation choice, not a portable capability order, and a router's allocation footprint is interpretable only against this measured action-value map.

Load-bearing premise

The adaptive and failure-aware results assume the verifier receives expected_total—the ground-truth frequency total—as part of its input, so the routing policies are trained on oracle correctness labels that a real deployment may not provide.

Editorial extensions

If this is right

  • A router must recalibrate the medium-to-strong action whenever the model pool or task contract changes; the same action can switch sign without any change in graph topology.
  • A small, constant number of strong calls can hold all-strong quality as workflows scale: in the homogeneous ladders, two strong calls per task preserved the quality regime while strong-call share fell from 20% to 3% as total calls grew from 10 to 66.
  • Allocation quality is three separable capabilities—coverage, selectivity, and evidence acquisition—and failing any one yields a distinct regime: under-coverage, over-allocation, or blocked exploration.
  • Sparse deployment can cut cost substantially: failure-aware allocation reached RMSE 1.512 at a $0.0662 test price versus all-strong RMSE 1.538 at $0.2533, a 73.9% price reduction in the same quality regime.
  • Structural predicates transfer across model pools even when the attached action does not; frozen transfer kept a stable footprint and a 5% strong-call share across training pools.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural extension the paper does not run is target-side action rebinding: keep the learned risk predicate but re-estimate the substitution action on the new pool; frozen transfer suggests the footprint would survive while the action would need new calibration.
  • The role-concentrated evidence implies a cheaper routing heuristic: spend strong-model budget almost entirely on merge and consolidation contexts, since solve-level swaps mostly tie locally; the paper reports the distribution but does not prescribe this as a deployment recipe.
  • Because the semantic reversal changes the task and contract together, the data do not yet say which ingredient—prompt, output schema, or outcome metric—drives the sign flip; a factorial decomposition would make that attribution.
  • If the oracle-verifier limitation is real, the fixed-tier calibration results still stand, but the adaptive allocation results should be read as upper bounds on what failure-aware routing can achieve without ground-truth verification.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 4 minor

Summary. The paper studies model substitution in multi-call LLM workflows. It proposes a predicate-action factorization that separates the decision of where to intervene (the risk predicate) from the model change to apply (the substitution action), and it measures node-conditioned substitution value through fixed-tier sweeps, input-matched probes, and allocation ablations. In a solve-merge-verify workflow spanning four model ladders, the paper reports that the same medium-to-strong substitution has different, sometimes opposite, value depending on model pool, workflow role, scale, and task. The authors introduce EvoCap, a trace-to-constraint allocator, and present a quality-price frontier together with ablations that separate coverage, selectivity, and evidence acquisition. The central conclusion is that tier labels do not define a portable capability order; substitution value must be calibrated per deployment.

Significance. If the empirical claims hold, the paper makes a useful conceptual contribution: it reframes model substitution as a deployment-conditioned action and provides a controlled experimental template for measuring role- and scale-dependent effects. The design has real strengths: deterministic task generation with disjoint held-out splits, three seeds per scale, exact-input matched probes that isolate role-level effects, and an auditable trace-to-constraint mechanism in EvoCap. The frozen-transfer and semantic-task experiments directly test portability, and the paper is careful to separate calibration, localization, and placement. However, the headline reversal of the Qwen-only ordering is not yet statistically supported, and the EvoCap results depend on a verifier signal whose informational status needs clarification. The core conceptual point may survive these concerns, but the specific numeric reversal that anchors the abstract is under-evidenced.

major comments (2)
  1. [§4.1, Table 1, Table 7] The central claim that the average Qwen-only ordering reverses is supported only by pooled means over 12 scale-seed cells, with no paired significance tests or confidence intervals. At the completed 64-call cell (Table 7), Qwen-only all-medium RMSE is 7.118 ± 1.047 and all-strong is 6.387 ± 1.184 (n = 3); the difference is 0.731, well within one standard deviation of either cell. The paper reports no seed-paired test across the four scales and no test of the ladder-by-tier interaction that would justify the "reverses" wording used in the abstract and §4.1. Please add paired cell-level tests (e.g., paired t-test or bootstrap over the 12 observations, and seed-paired per scale) with effect sizes and confidence intervals; if the reversal is not statistically robust, the abstract and §4.1 should be qualified accordingly.
  2. [S2, S3, §4.3] The EvoCap failure-aware updater consumes verifier ok/fail outcomes at inference to create escalation constraints (§S3), and the numeric VERIFY prompt supplies expected_total (§S2). Because expected_total is the true total of the target frequency vector, the verifier provides information that may not be available in a real deployment unless it is simply the known array length. The fixed-tier sweeps and input-matched probes do not use this channel, so the core conditional-value result is unaffected; however, the EvoCap-based placement and efficiency claims (§4.3, Figure S5, the "failure-aware" rows) depend on this signal. Please either state explicitly that expected_total is the known array length (a constant) and therefore not an oracle label, or provide an ablation of EvoCap without expected_total/verifier outcomes to show that the allocation results do not rely on privileged information.
minor comments (4)
  1. [Table 7] The table formatting is ambiguous: the column header "PolicynRMSE Strong calls/task" lacks separators, and entries such as "Qwen-only All-medium 37.118±1.0470.0" appear to concatenate the repetition count, RMSE, and strong-call share. Please reformat with clear column separators and align the reported values with the text in §4.1.
  2. [§4.2] The text refers to "Tables 2 and??" but the second table number is missing; please replace the placeholder with the correct supplementary table reference.
  3. [S2 vs §3.2.1] Section S2 states that the matched diagnostic re-executes "the first 16 arrays from the deterministic 32-task held-out stream for each seed," which would yield 48 arrays across three seeds, while §3.2.1 says "the same 48 held-out arrays." Please clarify whether the 48 arrays are 16 arrays per seed or a separate set of 48.
  4. [Table 8] The "Strong calls/task" column for all-strong in Table 8 reports 32.0, which appears to be the mean over the four scales (10, 18, 34, and 66 calls), not a per-scale count. Please state explicitly that these are pooled averages over the 12 scale-seed observations, or report the per-scale values.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the paper's action values and allocation results are direct measurements on held-out tasks, not outputs of a model fitted to those same outcomes.

full rationale

The paper's central derivation chain consists of fixed-tier sweeps (Eq. 2), input-matched interventions (Eq. 3), allocation ablations, homogeneous replication, frozen transfer, and the semantic task reversal. Each step is an empirical contrast over explicitly assigned tiers and held-out tasks, not a quantity derived by definition from the inputs. The risk weights, thresholds, and proxy formulas in Section S3 are stated to be hand-set engineering constants rather than fitted coefficients, so there is no fitted parameter later renamed as a prediction. The EvoCap policy is trained on disjoint deterministic tasks and evaluated on held-out tasks, which is standard evaluation rather than circularity. The verifier's expected_total is the fixed array length rather than the ground-truth frequency vector, and Section S2 states that the verifier does not receive the original array or full dictionary, so the allocator's validity signal is not an oracle for the RMSE metric. The central reversal claims are direct contrasts of measured workflow RMSE and exact-match values (Table 1, Figure 2, Figure 3), not consequences of how the predicates are defined. There are no load-bearing self-citations: EvoCap is introduced in this paper, and the cited literature is background. The absence of paired significance tests for some scale-resolved contrasts is a statistical-evidence concern, not a circularity concern. The derivation is therefore self-contained and shows no significant circularity.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central claim requires only the standard assumption that API models can be probed at temperature zero. The stronger practical claims additionally rely on hand-set EvoCap constants and on the verifier's access to ground-truth totals, which are the main free parameters and assumptions in this study.

free parameters (4)
  • EvoCap failure-risk weights = 0.15 to 0.55 (Table 3)
    Hand-set engineering constants that determine whether a node triggers escalation; they influence all EvoCap-based policy results but are not fit to data.
  • EvoCap dominant-feature thresholds = 0.60 fan-in, 0.65 disagreement, 0.68 entropy/depth/variance
    Hand-set cutoffs that convert risk scores into tier constraints in the allocator.
  • EvoCap confidence schedule = base 0.50/0.52, +0.08 per repeated evidence, cap 0.78
    Hand-set rule-conflict resolution constants that shape which constraints survive.
  • Semantic-difficulty proxy constants = e.g. 0.45 + 0.35d, 0.55 + min(0.35, N/32)
    Hand-set construction features for the semantic task that feed the risk predicate.
assumptions (4)
  • domain assumption The verify node can receive expected_total, the ground-truth total, as workflow input without invalidating routing evaluations.
    Invoked in S2 verifier prompt and used by EvoCap in S3; if ground truth is unavailable at inference, adaptive policies lose their correctness signal.
  • domain assumption Temperature-zero, single-draw runs are representative enough that three seeds capture the relevant outcome distribution.
    The paper pools 12 scale-seed cells and uses standard deviations, but does not test significance; overlapping error bars at 64 calls weaken this.
  • domain assumption Hand-set difficulty proxies ('entropy', 'variance', 'disagreement') are informative about which nodes benefit from a stronger model.
    S2 defines these as normalized construction features with no calibration evidence tying them to actual failure rates.
  • domain assumption A fixed solve-merge-verify skeleton on synthetic tasks generalizes to real multi-call workflows.
    The authors claim a practical sequence for large-scale routing while testing only two synthetic task contracts.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Beyond Tier Labels: Role- and Deployment-Dependent Model Substitution in Multi-Call LLM Workflows." pith.science (2026). https://pith.science/paper/TY7QT26V

@misc{pith2026260809155,
  author       = {Pith},
  title        = {Pith review of: Beyond Tier Labels: Role- and Deployment-Dependent Model Substitution in Multi-Call LLM Workflows},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TY7QT26V}},
  note         = {Machine review of arXiv:2608.09155}
}
read the original abstract

Large multi-call LLM systems pose a scientific problem that query-level routing does not capture: the value of a model depends on where it enters a dependent computation and on the deployment that surrounds that call. Existing routers typically decide \emph{where} to spend a stronger model while treating the benefit of the substitution itself as known. We separate these two decisions through a predicate-action factorization and evaluate it in controlled solve-merge-verify workflows spanning 8-64 solve calls and four three-tier model ladders. The resulting evidence reveals a consistent principle beneath apparently conflicting outcomes. On numeric frequency counting, all-strong reduces RMSE from 4.818 to 1.538 in the Mixed Qwen/GPT ladder, whereas the average Qwen-only ordering reverses. Input-matched interventions further show that the same medium-to-strong action has sharply different value across roles and scales. A semantic task-and-contract shift reverses the Mixed ordering again, while allocation ablations distinguish useful sparse placement from under-coverage and indiscriminate escalation. Together, these results establish model substitution as a deployment-conditioned action rather than a property implied by a tier label, and they provide a practical sequence for large-scale workflow routing: calibrate the action, resolve its role-conditioned effect, and then optimize its placement.

Figures

Figures reproduced from arXiv: 2608.09155 by the authors.

Figure 1
Figure 1. Overview of the proposed work. Stage 1 executes a fixed solve-merge-verify graph, while a risk predicate proposes node-level tier assignments. Stage 2 converts outcome traces into reusable constraints and then tests whether a substitution remains beneficial for the same candidate context after the deployment condition changes. the model change applied there. The link between them is learned from outcomes rather than… view at source ↗
Figure 2
Figure 2. Primary ladders produce different workflow-level tier orderings. [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. A complete task-and-contract shift reverses the Mixed workflow-level ordering. a [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Allocation ablations expose distinct routing failures. a [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

43 extracted references · 29 canonical work pages

  1. [1]

    Federate the router: Learning language model routers with sparse and decentralized evaluations.arXiv preprint arXiv:2601.22318, 2026

    Baris Askin, Shivam Patel, Anupam Nayak, et al. Federate the router: Learning language model routers with sparse and decentralized evaluations.arXiv preprint arXiv:2601.22318, 2026

    work page arXiv 2026

  2. [2]

    Frugalgpt: How to use large language models while reducing cost and improving performance.Transactions on Machine Learning Research, 2024

    Lingjiao Chen, Matei Zaharia, and James Zou. Frugalgpt: How to use large language models while reducing cost and improving performance.Transactions on Machine Learning Research, 2024

    work page 2024

  3. [3]

    Agentverse: Facilitating multi-agent collaboration and exploring emergent behaviors

    Weize Chen et al. Agentverse: Facilitating multi-agent collaboration and exploring emergent behaviors. In International Conference on Learning Representations, 2024

    work page 2024

  4. [4]

    Optima: Optimizing effectiveness and efficiency for LLM-based multi-agent system.arXiv preprint arXiv:2410.08115, 2024

    Weize Chen, Jiarui Yuan, Chen Qian, et al. Optima: Optimizing effectiveness and efficiency for LLM-based multi-agent system.arXiv preprint arXiv:2410.08115, 2024

    arXiv 2024

  5. [5]

    A unified approach to routing and cascading for llms

    Jasper Dekoninck, Maximilian Baader, and Martin Vechev. A unified approach to routing and cascading for llms. InProceedings of the 42nd International Conference on Machine Learning, volume 267 ofPMLR, pages 12987–13010, 2025

    work page 2025

  6. [6]

    Cascade-aware multi-agent routing: Spatio-temporal sidecars and geometry-switching

    Davide Di Gioia. Cascade-aware multi-agent routing: Spatio-temporal sidecars and geometry-switching. arXiv preprint arXiv:2603.17112, 2026

    work page arXiv 2026

  7. [7]

    Pay for hints, not answers: LLM shepherding for cost-efficient inference.arXiv preprint arXiv:2601.22132, 2026

    Ziming Dong, Hardik Sharma, Evan O’Toole, et al. Pay for hints, not answers: LLM shepherding for cost-efficient inference.arXiv preprint arXiv:2601.22132, 2026

    arXiv 2026

  8. [8]

    Glam: Efficient scaling of language models with mixture-of-experts

    Nan Du et al. Glam: Efficient scaling of language models with mixture-of-experts. InProceedings of the 39th International Conference on Machine Learning, volume 162 ofPMLR, 2022

    work page 2022

Show all 43 references
  1. [9]

    Improving factuality and reasoning in language models through multiagent debate

    Yilun Du et al. Improving factuality and reasoning in language models through multiagent debate. In Proceedings of the 41st International Conference on Machine Learning, 2024

    2024

  2. [10]

    Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity.Journal of Machine Learning Research, 23(120):1–39, 2022

    William Fedus, Barret Zoph, and Noam Shazeer. Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity.Journal of Machine Learning Research, 23(120):1–39, 2022

    2022

  3. [11]

    GraphPlanner: Graph memory-augmented agentic routing for multi-agent LLMs.arXiv preprint arXiv:2604.23626, 2026

    Tao Feng, Haozhen Zhang, Zijie Lei, et al. GraphPlanner: Graph memory-augmented agentic routing for multi-agent LLMs.arXiv preprint arXiv:2604.23626, 2026

    2026 arXiv

  4. [12]

    AgentScope: A flexible yet robust multi-agent platform.arXiv preprint arXiv:2402.14034, 2024

    Dawei Gao, Zitao Li, Xuchen Pan, et al. AgentScope: A flexible yet robust multi-agent platform.arXiv preprint arXiv:2402.14034, 2024

    2024 arXiv

  5. [13]

    Metagpt: Meta programming for a multi-agent collaborative framework

    Sirui Hong et al. Metagpt: Meta programming for a multi-agent collaborative framework. InInternational Conference on Learning Representations, 2024

    2024

  6. [14]

    Routerbench: A benchmark for multi-llm routing systems.arXiv preprint arXiv:2403.12031, 2024

    Qitian Hu et al. Routerbench: A benchmark for multi-llm routing systems.arXiv preprint arXiv:2403.12031, 2024

    2024 arXiv

  7. [15]

    Llm-blender: Ensembling large language models with pairwise ranking and generative fusion

    Dongfu Jiang, Xiang Ren, and Bill Yuchen Lin. Llm-blender: Ensembling large language models with pairwise ranking and generative fusion. InProceedings of the 61st Annual Meeting of the Association for Computational Linguistics, 2023

    2023

  8. [16]

    Kassem, Bernhard Schölkopf, and Zhijing Jin

    Aly M. Kassem, Bernhard Schölkopf, and Zhijing Jin. How robust are router-LLMs? analysis of the fragility of LLM routing capabilities.arXiv preprint arXiv:2504.07113, 2025

    2025 arXiv

  9. [17]

    Camel: Communicative agents for “mind” exploration of large scale language model society.Advances in Neural Information Processing Systems, 36, 2023

    Guohao Li et al. Camel: Communicative agents for “mind” exploration of large scale language model society.Advances in Neural Information Processing Systems, 36, 2023

    2023

  10. [18]

    More agents is all you need.Transactions on Machine Learning Research, 2024

    Junyou Li et al. More agents is all you need.Transactions on Machine Learning Research, 2024

    2024

  11. [19]

    Rethinking predictive modeling for LLM routing: When simple kNN beats complex learned routers.arXiv preprint arXiv:2505.12601, 2025

    Yang Li. Rethinking predictive modeling for LLM routing: When simple kNN beats complex learned routers.arXiv preprint arXiv:2505.12601, 2025

    2025 arXiv

  12. [20]

    CASTER: Breaking the cost-performance barrier in multi-agent orchestration via context-aware strategy for task efficient routing.arXiv preprint arXiv:2601.19793, 2026

    Shanyv Liu, Xuyang Yuan, Tao Chen, et al. CASTER: Breaking the cost-performance barrier in multi-agent orchestration via context-aware strategy for task efficient routing.arXiv preprint arXiv:2601.19793, 2026

    2026

  13. [21]

    A dynamic LLM-powered agent network for task-oriented agent collaboration

    Zijun Liu, Yanzhe Zhang, Peng Li, et al. A dynamic LLM-powered agent network for task-oriented agent collaboration. InFirst Conference on Language Modeling, 2024

    2024

  14. [22]

    Multi-agent ai systems need transparency.Nature Machine Intelligence, 8:1, 2026

    Nature Machine Intelligence. Multi-agent ai systems need transparency.Nature Machine Intelligence, 8:1, 2026. 10

    2026

  15. [23]

    Routellm: Learning to route llms with preference data

    Isaac Ong et al. Routellm: Learning to route llms with preference data. InInternational Conference on Learning Representations, 2025

    2025

  16. [24]

    One head, many models: Cross-attention routing for cost-aware LLM selection.arXiv preprint arXiv:2509.09782, 2025

    Roshini Pulishetty, Mani Kishan Ghantasala, Keerthy Kaushik Dasoju, et al. One head, many models: Cross-attention routing for cost-aware LLM selection.arXiv preprint arXiv:2509.09782, 2025

    2025 arXiv

  17. [25]

    Chatdev: Communicative agents for software development

    Chen Qian et al. Chatdev: Communicative agents for software development. InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics, pages 15174–15186, 2024

    2024

  18. [26]

    Scaling large language model-based multi-agent collaboration

    Chen Qian et al. Scaling large language model-based multi-agent collaboration. InInternational Conference on Learning Representations, 2025

    2025

  19. [27]

    Mixture-of-depths: Dynamically allocating compute in transformer-based language models.arXiv preprint arXiv:2404.02258, 2024

    David Raposo et al. Mixture-of-depths: Dynamically allocating compute in transformer-based language models.arXiv preprint arXiv:2404.02258, 2024

    2024 arXiv

  20. [28]

    Archon: An architecture search framework for inference-time techniques.arXiv preprint arXiv:2409.15254, 2024

    Jon Saad-Falcon, Adrian Gamarra Lafuente, Shlok Natarajan, et al. Archon: An architecture search framework for inference-time techniques.arXiv preprint arXiv:2409.15254, 2024

    2024 arXiv

  21. [29]

    Confident adaptive language modeling

    Tal Schuster et al. Confident adaptive language modeling. InAdvances in Neural Information Processing Systems, volume 35, 2022

    2022

  22. [30]

    C3PO: Optimized large language model cascades with probabilistic cost constraints for reasoning

    Antonios Valkanas, Soumyasundar Pal, Pavel Rumiantsev, et al. C3PO: Optimized large language model cascades with probabilistic cost constraints for reasoning. InAdvances in Neural Information Processing Systems, volume 38, 2025

    2025

  23. [31]

    Mixture-of-agents enhances large language model capabilities

    Junlin Wang et al. Mixture-of-agents enhances large language model capabilities. InInternational Conference on Learning Representations, 2025

    2025

  24. [32]

    Agentdropout: Dynamic agent elimination for token-efficient and high- performance llm-based multi-agent collaboration

    Zhenhailong Wang et al. Agentdropout: Dynamic agent elimination for token-efficient and high- performance llm-based multi-agent collaboration. InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics, 2025

    2025

  25. [33]

    Webb, Suchol Mondal, and Ida Momennejad

    Taylor W. Webb, Suchol Mondal, and Ida Momennejad. A brain-inspired agentic architecture to improve planning with llms.Nature Communications, 16:8633, 2025

    2025

  26. [34]

    AutoGen: Enabling next-gen LLM applications via multi-agent conversation.arXiv preprint arXiv:2308.08155, 2023

    Qingyun Wu, Gagan Bansal, Jieyu Zhang, et al. AutoGen: Enabling next-gen LLM applications via multi-agent conversation.arXiv preprint arXiv:2308.08155, 2023

    2023 arXiv

  27. [35]

    Magic: Investigation of large language model powered multi-agent in cognition, adaptability, rationality and collaboration

    Lin Xu et al. Magic: Investigation of large language model powered multi-agent in cognition, adaptability, rationality and collaboration. InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, 2024

    2024

  28. [36]

    R2-Router: A new paradigm for LLM routing with reasoning.arXiv preprint arXiv:2602.02823, 2026

    Jiaqi Xue, Qian Lou, Jiarong Xing, and Heng Huang. R2-Router: A new paradigm for LLM routing with reasoning.arXiv preprint arXiv:2602.02823, 2026

    2026 arXiv

  29. [37]

    Bamas: Structuring budget-aware multi-agent systems

    Jianing Yang et al. Bamas: Structuring budget-aware multi-agent systems. InProceedings of the AAAI Conference on Artificial Intelligence, volume 40, 2026

    2026

  30. [38]

    MasRouter: Learning to route LLMs for multi-agent systems.arXiv preprint arXiv:2502.11133, 2025

    Yanwei Yue, Guibin Zhang, Boyang Liu, et al. MasRouter: Learning to route LLMs for multi-agent systems.arXiv preprint arXiv:2502.11133, 2025

    2025 arXiv

  31. [39]

    When does multi-agent RL improve LLM workflows? workflow, scale, and policy-sharing tradeoffs.arXiv preprint arXiv:2605.24202, 2026

    Yifan Zeng, Yiran Wu, Yaolun Zhang, et al. When does multi-agent RL improve LLM workflows? workflow, scale, and policy-sharing tradeoffs.arXiv preprint arXiv:2605.24202, 2026

    2026 arXiv

  32. [40]

    Reinforcement learning for LLM-based multi-agent systems through orchestration traces

    Chenchen Zhang. Reinforcement learning for LLM-based multi-agent systems through orchestration traces. arXiv preprint arXiv:2605.02801, 2026

    2026 arXiv

  33. [41]

    RerouteGuard: Understanding and mitigating adversarial risks for LLM routing.arXiv preprint arXiv:2601.21380, 2026

    Wenhui Zhang, Huiyu Xu, Zhibo Wang, et al. RerouteGuard: Understanding and mitigating adversarial risks for LLM routing.arXiv preprint arXiv:2601.21380, 2026

    2026

  34. [42]

    Agent-as-a-router: Agentic model routing for coding tasks

    Pengfei Zhou, Zhiwei Tang, Yixing Ma, et al. Agent-as-a-router: Agentic model routing for coding tasks. arXiv preprint arXiv:2606.22902, 2026

    2026 arXiv

  35. [43]

    entropy”, “variance

    Mingchen Zhuge et al. Gptswarm: Language agents as optimizable graphs. InProceedings of the 41st International Conference on Machine Learning, 2024. 11 A Supplementary Information S1. Policy definitions and execution protocol All policies share the same solve–merge–verify runn...

    2024

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.