Pith. sign in

REVIEW 4 major objections 5 minor 61 references

Treating each LLM agent role call as a reliability-constrained choice over candidate instances, and hedging across them, matches near-full-parallel robustness at lower token and latency cost.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 13:03 UTC pith:BC3JMDXV

load-bearing objection HACO is a genuinely useful framing—adaptive hedge-set selection for LLM role invocation—with honest caveats, but the target-reliability guarantee rests on unverified calibration and independence assumptions. the 4 major comments →

arxiv 2607.19215 v1 pith:BC3JMDXV submitted 2026-07-21 cs.NI

HACO: Hedged Agent Computing for Reliable LLM Systems

Enhan Li , Hongyang Du This is my paper
classification cs.NI
keywords hedged agent computingreliability-constrained selectionLLM multi-agent systemsredundancy allocationcross-region inferenceconservative stopping ruleexperience harvestingruntime reliability control
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper argues that in deployed LLM agent systems, reliability is set less by which role or model a workflow names than by which concrete agent instance — model, region, and network path together — actually executes each role call, and that this choice can and should be made at runtime. It proposes HACO, which for each role invocation selects a hedge set of candidate instances using two deliberately opposed estimates: an optimistic score (estimated quality plus an uncertainty bonus, discounted by failure probability and latency) that decides ranking, and a conservative success estimate that decides when to stop adding candidates, once the accumulated success probability reaches a target τ. The central claim is that this adaptive redundancy approaches the robustness of running every candidate in parallel while spending far fewer tokens and less latency, and that the traces harvested from the redundant runs continuously improve future selections. If correct, HACO gives agent operators a single tunable reliability knob and a way to keep long-horizon workflows alive through region-level network and service failures. The paper supports the claim with experiments on three data-analysis benchmarks, under injected network, candidate, and evaluator degradation, plus a real multi-region deployment with injected failures.

Core claim

The paper's central claim: role-invocation redundancy is a tractable control problem — choose a hedge set S of candidates minimizing cost subject to P(qualified success | S) ≥ τ. The solution splits the estimate in two: an optimistic ranking utility U_i^+ decides which candidates get tried; a conservative reliability R_i^- decides only when to stop adding them. Under the paper's assumptions this stopping rule certifies the target reliability. The experiments show the payoff: hedge sets shift away from degraded zones, all-failed-step rates stay near zero where single-route baselines jump 67–90 points, and token and latency costs stay far below full-pool execution.

What carries the argument

The central object is the candidate agent instance — a role type, an LLM, and an execution environment taken as one choice unit. The mechanism is dual-bound coordination: candidates are ranked by the optimistic utility U_i^+ = min(1, μ_i + λσ_i)·ρ_i/D_i (quality posterior mean plus an exploration bonus, discounted by physical reliability and a logarithmic latency cost), while the hedge set is grown using the conservative estimate R_i^- = min(0.98, max(ε_R, μ_i − γσ_i))·ρ_i, multiplying failure probabilities Pfail ← Pfail(1 − R_i^-) until 1 − Pfail ≥ τ. Experience harvesting closes the loop: traces from each executed candidate update the posteriors and link profiles, so redundancy buys immedi

Load-bearing premise

The certificate that the hedge set reaches the target reliability (Prop. D.2) rests on R_i^- being a genuine lower bound on each candidate's true qualified-success probability and on candidates failing independently — yet R_i^- is a heuristic confidence bound with no calibration procedure given, and candidates that share a region are likely to fail together in exactly the outages the paper cites as motivation.

What would settle it

Run the paper's own procedure with a fixed target (say τ = 0.9) over many role invocations while periodically failing every candidate in one region simultaneously, without recalibrating R_i^-; if the measured fraction of invocations achieving qualified success falls below τ, the independence assumption behind the stopping certificate is violated. A second check: pick a volatile candidate observed only a few times, so the Beta posterior understates uncertainty and R_i^- overstates its true success probability, and see whether realized success over many trials stays above τ.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • The reliability target τ becomes a direct operating knob: in the paper's MatplotBench sweep, raising τ from 0.70 to 0.95 cuts step failure rate from 1.87% to 0.17% while token cost rises smoothly, letting operators choose where to sit on the cost–reliability frontier.
  • Degraded paths self-correct without a separate failure detector: because every hedge run harvests traces, HACO's zone mix shifts away from failing regions (Global selections down 80 points under network degradation), so robustness emerges from the allocation loop itself.
  • The method does not depend on a strong evaluator: replacing the router/judge with a small local model had only a minor effect, so the gains are not an artifact of a powerful LLM-as-judge.
  • HACO sits under any controller that emits role invocation events, and it leaves room for output aggregation (Best-of-N style selection or synthesis) as a downstream policy — the hedge problem and the selection problem are separate, composable layers.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The optimistic-ranking / conservative-stopping split is a transferable template: any reliability-constrained selection problem with uncertain quality estimates — LLM cascade depth, multi-cloud replication, edge task dispatch — could use the same device to avoid both under-exploring strong newcomers and closing the set too early.
  • The paper's motivating failures (cable cuts, thermal events) are correlated, region-wide events, which is precisely where its independence-based certificate is unproven; testing HACO with a zone-level common-cause failure model would show whether the hedge set needs correlation-aware inflation to keep the τ guarantee.
  • The paper flags (Appendix D.2) that its certificate depends on calibration and conditional independence, but gives no procedure to calibrate R_i^-; comparing predicted R_i^- values against observed success frequencies from harvested traces would turn the guarantee into a measurement-backed one — or reveal where it fails.
  • Since σ_i shrinks as candidates accumulate observations, the fixed exploration bonus λ and conservative margin γ effectively decay over time — an unstated behavior that suggests automatic scheduling of these coefficients (or of τ) as a natural follow-up.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes HACO, a runtime control scheme for LLM-based agent systems that treats each role invocation as a reliability-constrained selection problem over a hedge set of candidate agent instances, each coupling a role type, an LLM, and an execution environment. HACO ranks candidates by an optimistic utility U_i^+ and adds candidates until a conservative reliability accumulation, based on R_i^- and the product rule in Eq. (12), reaches a target system reliability τ. It then executes the hedge set in parallel, selects a winner among successful candidates, and harvests execution traces to update Beta capability profiles and network-link statistics. The paper reports experiments on MatplotBench, InfiAgent-Bench, and DSBench, including degradation studies and a real Azure deployment, and claims improved robustness and output quality at lower token and latency cost than exhaustive parallel execution.

Significance. If the reliability certificate were fully established, HACO would be a useful and fairly novel contribution to LLM agent systems: it converts redundancy from a fixed full-pool mechanism into an adaptive, query-dependent control variable with an explicit reliability/efficiency trade-off. The paper provides a clearly stated conditional proof (Prop. D.2), a detailed system description, a reasonable baseline set, and extensive evaluations including correlated degradation scenarios and a real Azure-backed deployment. The ablation study (Fig. 6(d)) also isolates the roles of environment-aware modeling, optimistic exploration, and conservative stopping. The contribution is weakened, however, by the fact that the two premises of the central certificate are not verified and by the absence of repeated-run uncertainty in the empirical claims.

major comments (4)
  1. [Eq. (11) and Appendix D.2] The central reliability certificate (Prop. D.2) assumes that R_i^- is a calibrated lower bound on the true qualified-success probability of each candidate. R_i^- as defined in Eq. (11) is min(0.98, max(epsilon_R, mu_i - gamma sigma_i) * rho_i), where mu_i/sigma_i are the posterior mean and standard deviation of a Beta distribution over normalized LLM-judge quality scores and rho_i is a point estimate of physical reliability. No formal definition of 'qualified success' is given, no threshold links the normalized quality score to that event, and no calibration or coverage experiment shows that R_i^- <= p_i with high probability. Appendix D.2 itself concedes that the certificate is conditional and that 'empirical monitoring or recalibration is required' when assumptions are violated, but no such monitoring or recalibration is provided or evaluated. Therefore the abstract and Section 4.2 cla
  2. [Eq. (6), Eq. (12), Table 3] The reliability accumulation in Eq. (12) multiplies (1 - R_i^-) across candidates, which is only valid under conditional independence. Eq. (6) explicitly states that conditional independence is assumed for tractability. However, the paper's motivating failure modes — Red Sea cable cuts and the Azure West Europe thermal event (Section 1) — are region-wide correlated events, and the paper's own degradation experiments (Table 3: Global-link outage and Global candidate outage) induce en-bloc failures of all Global candidates. Under correlated failures, the product in Eq. (12) overstates the true success probability of the hedge set, so the stopping rule can terminate at a set whose actual success probability is below τ. Appendix D.2 notes this gap, but the paper does not quantify the impact of correlation or propose a correction. At minimum, the authors should evaluate the difference between
  3. [Section 4.2, Table 7, Fig. 6] All reported numerical results appear to come from a single run per configuration. The experiments include stochastic network delays, jitter, message-level loss, and probabilistic candidate failures, so the reported differences (e.g., MatplotBench average score 58.86 for HACO vs. 47.75 for BestOne in Table 7) may not be statistically stable. The reliability-target sweep in Fig. 6 reports failure counts like 11/587 and 1/596 without confidence intervals, and the degradation plots in Fig. 3 have no error bands. Since the paper's central claim is an empirical improvement in robustness and quality, the authors should provide repeated-seed results with standard deviations, confidence intervals, or significance tests.
  4. [Appendix E.4, Table 8] The 'execution environment-adjusted reference' to external baselines is not a direct measurement. The adjustment is derived from two anchor points and then applied to previously published scores, but the calibration procedure is under-specified and the resulting adjusted numbers (e.g., 52.61, 52.0, 50.1) are estimates, not observations. This table should not be used as evidence that HACO outperforms external systems. It should either be removed from the evidence chain or replaced by re-running the external baselines in the same harness with the same execution environment.
minor comments (5)
  1. [Eq. (15)] Winner selection uses the optimistic utility U_i^+ among successful candidates, which includes an estimated quality term, a latency discount, and uncertainty bonus. The paper claims output-quality improvement, but the selected output is not necessarily the one with highest actual quality. Clarify the relationship between U_i^+ and measured output quality, and whether the reported scores are based on the selected winner or on the best available successful output.
  2. [Eq. (7)] The term 'bandwidth jitter' j_i in Eq. (7) is not defined in the network simulator (Appendix C.2.1), which models latency jitter, not bandwidth jitter. The units and the effect of kappa on effective bandwidth should be clarified, since this term influences the cost model.
  3. [Table 8] The row for 'DATAWISE w/ visual tool [54]' contains three numbers (64.33, 52.61, 58.86) with the parenthetical '(w/o HACO) (w/ HACO)' but the alignment in the table is ambiguous. It should be made clear which number is the idealized reference, which is the adjusted reference, and which is the HACO result.
  4. [Appendix C.2.1] Eq. (18) clips path delay with max(T_min, mu + xi); with a Gaussian jitter this is fine, but the paper should state that xi is truncated to avoid negative latencies, as the current notation implies raw Gaussian noise is used and then floored.
  5. [Eq. (1)] The formulation in Eq. (1) presents an optimization problem (minimize cost subject to reliability), but the algorithm is a greedy prefix rule under the U_i^+ ordering. Prop. D.3 only shows that HACO returns the shortest feasible prefix of that ordering; it does not show cost-optimality. The text should avoid implying that Eq. (1) is solved exactly, or state that HACO is a heuristic.

Circularity Check

0 steps flagged

No significant circularity: the central claims rest on external benchmarks and a clearly conditional certificate, not on self-citation or fitted inputs renamed as predictions.

full rationale

The paper's derivation chain is not circular under the review standard. Phase 2 defines R_i^- as a conservative capability-reliability product (Eq. 11), accumulates it (Eq. 12), and stops when 1 - P_fail >= tau (Eq. 13). Proposition D.2 then derives P(qualified success | S) >= tau only under explicit assumptions that R_i^- are calibrated lower bounds on the true qualified-success probabilities and that candidates are conditionally independent. This is a conditional mathematical property, not a restatement of the empirical conclusion, and the paper itself disclaims it: "When the assumptions are violated, the certificate should be viewed as an approximate reliability estimate, and empirical monitoring or recalibration is required" (Appendix D.2). The main empirical claims are supported by comparisons against external benchmarks and degradation settings (Figs. 3-6), not derived from the fitted R_i^- values. The external-reference table (Table 8) is calibrated using the reported DATAWISE score and a measured fixed-zone anchor, while the HACO row is directly observed, so it does not smuggle HACO's own outputs into the comparison. No load-bearing self-citations appear, and no ansatz is imported through the authors' prior work. The unvalidated calibration and independence assumptions are genuine correctness/robustness risks, but they are not circularity.

Axiom & Free-Parameter Ledger

5 free parameters · 5 axioms · 0 invented entities

The method's reliability guarantee is conditional on calibration and independence assumptions; the main contribution is algorithmic and empirical. All load-bearing assumptions are listed above.

free parameters (5)
  • tau (target system reliability) = 0.9 default; swept 0.70–0.95
    Operating constraint that controls hedge-set size; user-specified, not fitted, but has direct effect on the reliability claim.
  • lambda (exploration coefficient) = 1.1
    Controls the optimism bonus in U_i^+ (Eq. 10); hand-chosen, not derived.
  • gamma (conservative margin) = 0.7
    Controls the lower-confidence capability estimate in R_i^- (Eq. 11); hand-chosen and load-bearing for the stopping rule.
  • kappa (bandwidth jitter penalty) = 2.0
    Shapes effective latency in Eq. 7; hand-chosen.
  • eta (latency discount strength) = 0.5
    Sets the severity of the latency discount in Eq. 9; hand-chosen.
axioms (5)
  • domain assumption R_i^- is a calibrated lower bound on true qualified-success probability
    Required for Prop. D.2's reliability certificate; not established experimentally. Eq. (11) gives a heuristic percentile times estimated reliability.
  • domain assumption Conditional independence of candidate success/failure given observed runtime state
    Used to multiply (1-R_i^-) in Eq. (12); likely violated during region-wide outages or correlated model failures.
  • domain assumption Execution success probability factorizes as (1-f_int)(1-l_net)
    Eq. (6) assumes internal and network failures are independent and that no other failure modes matter.
  • domain assumption LLM-judge quality score q_i is an unbiased normalized output-quality signal
    Beta updates (Eq. 4) and winner selection (Eq. 15) rely on q_i; evaluator degradation is tested, but bias and calibration are not addressed.
  • domain assumption Latency model in Eq. (7) plus Weber-Fechner cost in Eq. (8) captures user-perceived cost
    Design choice for the cost term; not independently validated.

pith-pipeline@v1.3.0-alltime-deepseek · 26211 in / 10928 out tokens · 110867 ms · 2026-08-01T13:03:46.056733+00:00 · methodology

0 comments
read the original abstract

As large language model (LLM) agents move from isolated prompting to longhorizon workflows, failures increasingly arise at the role-to-instance binding boundary, where task-specific role requests must be assigned to concrete agent instances under current service, network, and query conditions. Existing agent system research has improved role specialization, workflow topology, memory, and tool use, but often assumes a fixed stable execution environment. This assumption limits deployed reliability, because the same role request can exhibit different latency, failure probability, and output quality across agent instances operating under different service regions and network conditions. We propose Hedged Agent Computing (HACO), a runtime control scheme that treats each role request as a reliability-constrained selection problem over candidate agent instances, each coupling a role type, an LLM, and a concrete execution environment. Different from routing, HACO adaptively selects a hedge set of candidates for each invocation. Its allocation rule combines optimistic ranking, which prioritizes candidates with high estimated quality, reliability, and informative uncertainty, with conservative reliability accumulation, which stops selection only after the hedge set reaches a target success probability. Through experience harvesting, HACO updates candidate and link profiles from all executed candidate traces, including quality, success, latency, and network statistics. Experiments on various benchmarks, together with runtime degradation studies, show that HACO improves robustness and output quality under changing deployment conditions, while using lower token and latency cost than exhaustive parallel execution.

Figures

Figures reproduced from arXiv: 2607.19215 by Enhan Li, Hongyang Du.

Figure 1
Figure 1. Figure 1: Three coupled axes of role invocation in MAS. LLM-based agent systems, especially multi-agent systems (MAS), have become a common paradigm for complex task execution and advanced automation [24, 41, 17, 51]. Compared with single-LLM execution, agent systems can distribute a task across specialized role types, coordinate intermediate outputs, cross-check partial results, and repair errors through iterative … view at source ↗
Figure 2
Figure 2. Figure 2: Overview of the HACO paradigm. 2 Related Work and Theory Motivation Agent harnesses. Recent work increasingly treats LLM agents as complete execution systems supported by agent harnesses, where the surrounding layer manages context construction, memory access, tool invocation, validation, and lifecycle control [4, 45, 57, 33, 42]. This harness view shows that practical agent performance depends on system m… view at source ↗
Figure 3
Figure 3. Figure 3: Reliability evaluation under runtime degradation and real deployment failures. Rows correspond to network degradation, candidate degradation, evaluator degradation, and a real-world Azure-backed deployment with injected candidate failures. 21.1 61.7 9.0 38.0 -6.9 329 14.4 221 -0.7 10.1 Avg. Score Strict Succ. (%) Latency Eff. Token Eff. Reliability MatplotBench -2.4 94.5 -2.1 90.8 -0.9 219 3.7 142 -1.1 15.… view at source ↗
Figure 4
Figure 4. Figure 4: Performance comparison under the heterogeneous execution environment. Results are reported on MatplotBench, InfiAgent-Bench, and DSBench for Random, BestOne, MoA-style Best-of-N, fixed-zone Local, fixed-zone Regional, fixed-zone Global, and HACO. tasks such as InfiAgent-Bench. The selected-zone distribution in [PITH_FULL_IMAGE:figures/full_fig_p008_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: HACO behavior. (a) Mean hedge size of baselines and HACO, (b) per-task role redundancy, and (c) selected-zone frequency across three benchmarks. 0.70 0.80 0.90 0.95 Reliability τ 0.00 0.01 0.02 Rate 11/587 4/614 2/560 1/596 (a) Step Failure Rate 0.70 0.80 0.90 0.95 Reliability τ 0 20 40 60 Score 53.77 53.90 57.02 58.86 (b) Average Score 60k 90k 120k Token Mean 52 54 56 58 60 Score Mean τ=0.70 τ=0.80 τ=0.90… view at source ↗
Figure 6
Figure 6. Figure 6: Reliability-target sweep and component ablation in MatplotBench. Panels (a)–(c) sweep the system reliability target τ ∈ {0.70, 0.80, 0.90, 0.95} and report step failure rate, average score, and the score-token relationship. Panel (d) compares HACO with ablated variants that remove environment-aware modeling (Env), the uncertainty bonus (UCB), or conservative stopping (LCB). but useful candidates. Replacing… view at source ↗
Figure 7
Figure 7. Figure 7: Role of redundancy in HACO. Redundant or hedged execution improves reliability for the current role invocation while also collecting execution traces for future allocation. HACO records quality, success/failure, latency, and network statistics from executed candidates, updates candidate and link profiles, and uses these profiles to improve later routing and hedge-set selection. C Experimental Setup Details… view at source ↗
Figure 8
Figure 8. Figure 8: Role-oriented view of the DATAWISE concurrent FSM workflow. The underlying controller is a six-state finite-state machine, while the execution dynamics can be organized into four role types: planner, coder, debugger, and filter. The figure highlights role collaboration and the self-debug recovery path. C.2 Simulating Environment [PITH_FULL_IMAGE:figures/full_fig_p017_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: provides an overview of the simulated experimental setup. It combines the zone-aware network topology over Local, Regional, and Global execution zones with the role-specific candidate pool used by the MAS. The network side specifies latency, jitter, bandwidth, and loss for each logical inter-zone link, while the candidate side shows the LLM backbone, replica count, failure rates, and temperature settings f… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

61 extracted references · 12 linked inside Pith

  1. [1]

    Increase throughput with cross-region inference

    Amazon Web Services. Increase throughput with cross-region inference. https://docs.aws. amazon.com/bedrock/latest/userguide/cross-region-inference.html , 2026. Ac- cessed: 2026-03-13

    2026

  2. [2]

    Geographic cross-region inference

    Amazon Web Services. Geographic cross-region inference. https://docs.aws.amazon. com/bedrock/latest/userguide/geographic-cross-region-inference.html ,

  3. [3]

    Regional availability - amazon bedrock

    Amazon Web Services. Regional availability - amazon bedrock. https://docs.aws.amazon. com/bedrock/latest/userguide/models-region-compatibility.html , 2026. Ac- cessed: 2026-03-13

    2026

  4. [4]

    Effective harnesses for long-running agents

    Anthropic. Effective harnesses for long-running agents. Anthropic Engi- neering Blog, 2025. URL https://www.anthropic.com/engineering/ effective-harnesses-for-long-running-agents. Published Nov. 26, 2025

    2025

  5. [5]

    Yuda Bi, Ying Zhu, and Vince D. Calhoun. Redundancy as a structural information principle for learning and generalization.arXiv preprint arXiv:2510.10938, 2025. URL https://arxiv. org/abs/2510.10938

    arXiv 2025

  6. [6]

    Nist net: A linux-based network emulation tool.Computer Communication Review, 33(3):111–126, 2003

    Mark Carson and Darrin Santay. Nist net: A linux-based network emulation tool.Computer Communication Review, 33(3):111–126, 2003

    2003

  7. [7]

    A-mapreduce: Executing wide search via agentic mapreduce.arXiv preprint arXiv:2602.01331, 2026

    Mingju Chen, Guibin Zhang, Heng Chang, Yuchen Guo, and Shiji Zhou. A-mapreduce: Executing wide search via agentic mapreduce.arXiv preprint arXiv:2602.01331, 2026

    arXiv 2026

  8. [8]

    Guiding principles for excellent generative ai architecture design, Septem- ber 2025

    China Academy of Information and Communications Technology and Alibaba Cloud Com- puting Co., Ltd. Guiding principles for excellent generative ai architecture design, Septem- ber 2025. URL https://pdf.dfcfw.com/pdf/H3_AP202509181745854315_1.pdf. Ac- cessed: 2026-04-07

    2025

  9. [9]

    Cloudflare global network | data center locations

    Cloudflare. Cloudflare global network | data center locations. https://www.cloudflare. com/network/, 2026. Accessed: 2026-03-13

    2026

  10. [10]

    Cloudflare workers ai overview

    Cloudflare. Cloudflare workers ai overview. https://developers.cloudflare.com/ workers-ai/, 2026. Accessed: 2026-03-13

    2026

  11. [11]

    Microsoft says azure cloud computing service disrupted by fiber cuts in the red sea

    CNBC. Microsoft says azure cloud computing service disrupted by fiber cuts in the red sea. https://www.cnbc.com/2025/09/06/ microsoft-azure-cloud-computing-service-disrupted-red-sea-fiber-cuts. html, 2025. Accessed: 2026-05-02

    2025

  12. [12]

    Red sea cables have been damaged, disrupting internet traffic

    CNN Business. Red sea cables have been damaged, disrupting internet traffic. https: //www.cnn.com/2024/03/04/business/red-sea-cables-cut-internet , 2024. Ac- cessed: 2026-05-02

    2024

  13. [13]

    Hybrid LLM: Cost-efficient and quality- aware query routing.arXiv preprint arXiv:2404.14618, 2024

    Dujian Ding, Ankur Mallick, Chi Wang, Robert Sim, Subhabrata Mukherjee, Victor Ruhle, Laks VS Lakshmanan, and Ahmed Hassan Awadallah. Hybrid LLM: Cost-efficient and quality- aware query routing.arXiv preprint arXiv:2404.14618, 2024

    Pith/arXiv arXiv 2024

  14. [14]

    Agentswing: Adaptive parallel context management routing for long-horizon web agents.arXiv preprint arXiv:2603.27490, 2026

    Zhaopeng Feng, Liangcai Su, Zhen Zhang, Xinyu Wang, Xiaotian Zhang, Xiaobin Wang, Runnan Fang, Qi Zhang, Baixuan Li, Shihao Cai, et al. Agentswing: Adaptive parallel context management routing for long-horizon web agents.arXiv preprint arXiv:2603.27490, 2026

    arXiv 2026

  15. [15]

    Beta regression for modelling rates and proportions

    Silvia Ferrari and Francisco Cribari-Neto. Beta regression for modelling rates and proportions. Journal of applied statistics, 31(7):799–815, 2004

    2004

  16. [16]

    Regions - fireworks ai docs

    Fireworks AI. Regions - fireworks ai docs. https://docs.fireworks.ai/deployments/ regions, 2026. Accessed: 2026-03-13. 10

    2026

  17. [17]

    Magentic-one: A generalist multi-agent system for solving complex tasks

    Adam Fourney, Gagan Bansal, Hussein Mozannar, Cheng Tan, Eduardo Salinas, Erkang Zhu, Friederike Niedtner, Grace Proebsting, Griffin Bassman, Jack Gerrits, Jacob Alber, Peter Chang, Ricky Loynd, Robert West, Victor Dibia, Ahmed Awadallah, Ece Kamar, Rafah Hosn, and Saleema Amershi. Magentic-one: A generalist multi-agent system for solving complex tasks. a...

    Pith/arXiv arXiv 2024

  18. [18]

    Deployments and endpoints

    Google Cloud. Deployments and endpoints. https://docs.cloud.google.com/ vertex-ai/generative-ai/docs/learn/locations, 2026. Accessed: 2026-03-13

    2026

  19. [19]

    Standard paygo

    Google Cloud. Standard paygo. https://docs.cloud.google.com/vertex-ai/ generative-ai/docs/standard-paygo, 2026. Accessed: 2026-03-13

    2026

  20. [20]

    Kollaps: Decentralized and dynamic topology emulation.arXiv preprint arXiv:2004.02253, 2020

    Paulo Gouveia, João Neves, Carlos Segarra, Luca Liechti, Shady Issa, Valerio Schiavoni, and Miguel Matos. Kollaps: Decentralized and dynamic topology emulation.arXiv preprint arXiv:2004.02253, 2020

    Pith/arXiv arXiv 2004

  21. [21]

    Language model cascades: Token-level uncertainty and beyond

    Neha Gupta, Harikrishna Narasimhan, Wittawat Jitkrittum, Ankit Singh Rawat, Aditya Krishna Menon, and Sanjiv Kumar. Language model cascades: Token-level uncertainty and beyond. arXiv preprint arXiv:2404.10136, 2024

    Pith/arXiv arXiv 2024

  22. [22]

    LLM-based multi-agent systems for software engineering: Literature review, vision, and the road ahead.ACM Transactions on Software Engineering and Methodology, 34(5):1–30, 2025

    Junda He, Christoph Treude, and David Lo. LLM-based multi-agent systems for software engineering: Literature review, vision, and the road ahead.ACM Transactions on Software Engineering and Methodology, 34(5):1–30, 2025

    2025

  23. [23]

    Linux Network Emulator (NetEm)

    Stephen Hemminger.tc-netem(8) Linux Manual Page, 2005. Linux Network Emulator (NetEm)

    2005

  24. [24]

    Metagpt: Meta programming for a multi-agent collaborative framework.arXiv preprint arXiv:2308.00352, 2023

    Sirui Hong, Mingchen Zhuge, Jiaqi Chen, Xiawu Zheng, Yuheng Cheng, Ceyao Zhang, Jinlin Wang, Zili Wang, Steven Ka Shing Yau, Zijuan Lin, Liyang Zhou, Chenyu Ran, Lingfeng Xiao, Chenglin Wu, and Jürgen Schmidhuber. Metagpt: Meta programming for a multi-agent collaborative framework.arXiv preprint arXiv:2308.00352, 2023. URL https://arxiv.org/ abs/2308.00352

    Pith/arXiv arXiv 2023

  25. [25]

    Infiagent-dabench: Evaluating agents on data analysis tasks

    Xueyu Hu, Ziyu Zhao, Shuang Wei, et al. Infiagent-dabench: Evaluating agents on data analysis tasks. InInternational Conference on Machine Learning, 2024

    2024

  26. [26]

    RouterEval: A comprehensive benchmark for routing LLMs to explore model- level scaling up in LLMs.arXiv preprint arXiv:2503.10657, 2025

    Zhongzhan Huang, Guoming Ling, Yupei Lin, Yandong Chen, Shanshan Zhong, Hefeng Wu, and Liang Lin. RouterEval: A comprehensive benchmark for routing LLMs to explore model- level scaling up in LLMs.arXiv preprint arXiv:2503.10657, 2025

    Pith/arXiv arXiv 2025

  27. [27]

    LLM-blender: Ensembling large language models with pairwise ranking and generative fusion

    Dongfu Jiang, Xiang Ren, and Bill Yuchen Lin. LLM-blender: Ensembling large language models with pairwise ranking and generative fusion. InProceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 14165–14178, 2023

    2023

  28. [28]

    Dsbench: How far are data science agents from becoming data science experts? InInternational Conference on Learning Representations, 2025

    Liqiang Jing, Zhehui Huang, Xiaoyang Wang, Wenlin Yao, Wenhao Yu, Kaixin Ma, Hongming Zhang, Xinya Du, and Dong Yu. Dsbench: How far are data science agents from becoming data science experts? InInternational Conference on Learning Representations, 2025

    2025

  29. [29]

    Beta calibration: a well-founded and easily implemented improvement on logistic calibration for binary classifiers

    Meelis Kull, Telmo Silva Filho, and Peter Flach. Beta calibration: a well-founded and easily implemented improvement on logistic calibration for binary classifiers. InArtificial intelligence and statistics, pages 623–631. PMLR, 2017

    2017

  30. [30]

    Efficient memory management for large language model serving with PagedAttention

    Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph Gonzalez, Hao Zhang, and Ion Stoica. Efficient memory management for large language model serving with PagedAttention. InProceedings of the 29th symposium on operating systems principles, pages 611–626, 2023

    2023

  31. [31]

    Auto routing - litellm

    LiteLLM. Auto routing - litellm. https://docs.litellm.ai/docs/proxy/auto_ routing, 2026. Accessed: 2026-03-13

    2026

  32. [32]

    Beta-lr: Interpretable logical reasoning based on beta distribution

    Yizhuo Ma, Ke Qin, and Shuang Liang. Beta-lr: Interpretable logical reasoning based on beta distribution. InFindings of the Association for Computational Linguistics: NAACL 2024, pages 1945–1955, 2024. 11

    2024

  33. [33]

    A survey of context engineering for large language models

    Lingrui Mei, Jiayu Yao, Yuyao Ge, Yiwei Wang, Baolong Bi, Yujun Cai, Jiazhi Liu, Mingyu Li, Zhong-Zhi Li, Duzhen Zhang, et al. A survey of context engineering for large language models. arXiv preprint arXiv:2507.13334, 2025

    Pith/arXiv arXiv 2025

  34. [34]

    Understanding deployment types in microsoft foundry models

    Microsoft. Understanding deployment types in microsoft foundry models. https: //learn.microsoft.com/en-us/azure/foundry/foundry-models/concepts/ deployment-types, 2026. Accessed: 2026-03-13

    2026

  35. [35]

    Post incident review: Thermal event impacting multiple services – west europe

    Microsoft Azure. Post incident review: Thermal event impacting multiple services – west europe. https://azure.status.microsoft/en-us/status/history/?force_ isolation=true, 2025. Tracking ID: 2LGD-9VG. Accessed: 2026-05-02

    2025

  36. [36]

    Routellm: Learning to route llms with preference data

    Isaac Ong, Amjad Almahairi, Vincent Wu, Wei-Lin Chiang, Tianhao Wu, Joseph E Gonzalez, M Waleed Kadous, and Ion Stoica. Routellm: Learning to route llms with preference data. arXiv preprint arXiv:2406.18665, 2024

    Pith/arXiv arXiv 2024

  37. [37]

    Data controls in the openai platform

    OpenAI. Data controls in the openai platform. https://developers.openai.com/api/ docs/guides/your-data/, 2026. Accessed: 2026-03-13

    2026

  38. [38]

    Intelligent multi-provider request routing

    OpenRouter. Intelligent multi-provider request routing. https://openrouter.ai/docs/ guides/routing/provider-selection, 2026. Accessed: 2026-03-13

    2026

  39. [39]

    Openrouter quickstart guide

    OpenRouter. Openrouter quickstart guide. https://openrouter.ai/docs/quickstart,

  40. [40]

    Fallbacks - portkey docs

    Portkey. Fallbacks - portkey docs. https://portkey.ai/docs/product/ai-gateway/ fallbacks, 2026. Accessed: 2026-03-13

    2026

  41. [41]

    Accessed: 2026-03-13

    2026

  42. [42]

    Tool learning with large language models: A survey.Frontiers of Computer Science, 19(8):198343, 2025

    Changle Qu, Sunhao Dai, Xiaochi Wei, Hengyi Cai, Shuaiqiang Wang, Dawei Yin, Jun Xu, and Ji-Rong Wen. Tool learning with large language models: A survey.Frontiers of Computer Science, 19(8):198343, 2025

    2025

  43. [43]

    Chatdev: Communicative agents for software development.ACL, 2024

    Chen Qian, Wei Liu, Hongzhang Liu, Nuo Chen, Yufan Dang, Jiahao Li, Cheng Yang, Weize Chen, Yusheng Su, Xin Cong, Juyuan Xu, Dahai Li, Zhiyuan Liu, and Maosong Sun. Chatdev: Communicative agents for software development.ACL, 2024. URL https://arxiv.org/ abs/2307.07924

    Pith/arXiv arXiv 2024

  44. [44]

    The logarith- mic nature of qoe and the role of the weber-fechner law in qoe assessment

    Peter Reichl, Sebastian Egger, Raimund Schatz, and Alessandro D’Alconzo. The logarith- mic nature of qoe and the role of the weber-fechner law in qoe assessment. In2010 IEEE international conference on communications, pages 1–5. IEEE, 2010

    2010

  45. [45]

    Trism for agentic Trism: A review of trust, risk, and security management in LLM-based agentic multi-agent systems.AI Open, 2026

    Shaina Raza, Ranjan Sapkota, Manoj Karkee, and Christos Emmanouilidis. Trism for agentic Trism: A review of trust, risk, and security management in LLM-based agentic multi-agent systems.AI Open, 2026

    2026

  46. [46]

    SLoRA: Scalable serving of thousands of LoRA adapters.Proceedings of Machine Learning and Systems, 6:296–311, 2024

    Ying Sheng, Shiyi Cao, Dacheng Li, Coleman Hooper, Nicholas Lee, Shuo Yang, Christopher Chou, Banghua Zhu, Lianmin Zheng, Kurt Keutzer, et al. SLoRA: Scalable serving of thousands of LoRA adapters.Proceedings of Machine Learning and Systems, 6:296–311, 2024

    2024

  47. [47]

    The importance of agent harness in 2026

    Philipp Schmid. The importance of agent harness in 2026. Blog post, 2026. URL https: //www.philschmid.de/agent-harness-2026

    2026

  48. [48]

    Junlin Wang, Jue Wang, Ben Athiwaratkun, Ce Zhang, and James Y . Zou. Mixture-of-agents enhances large language model capabilities. InInternational Conference on Learning Represen- tations, 2025

    2025

  49. [49]

    Causal LLMs routing: End-to-end regret minimization from observational data.arXiv preprint arXiv:2505.16037, 2025

    Asterios Tsiourvas, Wei Sun, and Georgia Perakis. Causal LLMs routing: End-to-end regret minimization from observational data.arXiv preprint arXiv:2505.16037, 2025

    arXiv 2025

  50. [50]

    Autogen: Enabling next-gen llm applications via multi-agent conversations

    Qingyun Wu, Gagan Bansal, Jieyu Zhang, Yiran Wu, Beibin Li, Erkang Zhu, Li Jiang, Xiaoyun Zhang, Shaokun Zhang, Jiale Liu, et al. Autogen: Enabling next-gen llm applications via multi-agent conversations. InFirst conference on language modeling, 2024

    2024

  51. [51]

    Self-consistency improves chain of thought reasoning in language models.arXiv preprint arXiv:2203.11171, 2022

    Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models.arXiv preprint arXiv:2203.11171, 2022. 12

    Pith/arXiv arXiv 2022

  52. [52]

    Understanding agent scaling in llm-based multi-agent systems via diversity.arXiv preprint arXiv:2602.03794, 2026

    Yingxuan Yang, Chengrui Qu, Muning Wen, Laixi Shi, Ying Wen, Weinan Zhang, Adam Wierman, and Shangding Gu. Understanding agent scaling in llm-based multi-agent systems via diversity.arXiv preprint arXiv:2602.03794, 2026. URL https://arxiv.org/abs/2602. 03794

    arXiv 2026

  53. [53]

    Beyond self-talk: A communication-centric survey of LLM-based multi-agent systems.arXiv preprint arXiv:2502.14321, 2025

    Bingyu Yan, Zhibo Zhou, Litian Zhang, Lian Zhang, Ziyi Zhou, Dezhuang Miao, Zhoujun Li, Chaozhuo Li, and Xiaoming Zhang. Beyond self-talk: A communication-centric survey of LLM-based multi-agent systems.arXiv preprint arXiv:2502.14321, 2025

    Pith/arXiv arXiv 2025

  54. [54]

    Datawiseagent: A notebook-centric llm agent framework for automated data science.arXiv preprint arXiv:2503.07044, 2025

    Ziming You, Yumiao Zhang, Dexuan Xu, Yiwei Lou, Yandong Yan, Wei Wang, Huaming Zhang, and Yu Huang. Datawiseagent: A notebook-centric llm agent framework for automated data science.arXiv preprint arXiv:2503.07044, 2025

    arXiv 2025

  55. [55]

    Matplotagent: Method and evaluation for llm-based agentic scientific data visualization

    Zhiyu Yang, Zihan Zhou, Shuo Wang, et al. Matplotagent: Method and evaluation for llm-based agentic scientific data visualization. InFindings of ACL, 2024

    2024

  56. [56]

    Efficient contextual LLM cascades through budget-constrained policy learning.Advances in Neural Information Processing Systems, 37:91691–91722, 2024

    Xuechen Zhang, Zijian Huang, Ege Onur Taga, Carlee Joe-Wong, Samet Oymak, and Jiasi Chen. Efficient contextual LLM cascades through budget-constrained policy learning.Advances in Neural Information Processing Systems, 37:91691–91722, 2024

    2024

  57. [57]

    Large language model cas- cades with mixture of thoughts representations for cost-efficient reasoning.arXiv preprint arXiv:2310.03094, 2023

    Murong Yue, Jie Zhao, Min Zhang, Liang Du, and Ziyu Yao. Large language model cas- cades with mixture of thoughts representations for cost-efficient reasoning.arXiv preprint arXiv:2310.03094, 2023

    Pith/arXiv arXiv 2023

  58. [58]

    {DistServe}: Disaggregating prefill and decoding for goodput-optimized large language model serving

    Yinmin Zhong, Shengyu Liu, Junda Chen, Jianbo Hu, Yibo Zhu, Xuanzhe Liu, Xin Jin, and Hao Zhang. {DistServe}: Disaggregating prefill and decoding for goodput-optimized large language model serving. In18th USENIX Symposium on Operating Systems Design and Implementation (OSDI 24), pages 193–210, 2024. A Cross-Region Inference in Practice This section docume...

    2024

  59. [59]

    A survey on the memory mechanism of large language model-based agents

    Zeyu Zhang, Quanyu Dai, Xiaohe Bo, Chen Ma, Rui Li, Xu Chen, Jieming Zhu, Zhenhua Dong, and Ji-Rong Wen. A survey on the memory mechanism of large language model-based agents. ACM Transactions on Information Systems, 43(6):1–47, 2025

    2025

  60. [61]

    Enhance Agent Runtime Reliability Immediate effect Success (Fast) Success (Normal) Failed (Error) Slow (High Latency) Reliable Response Survives failures Reduces outage risk Improves success under unstable network / service conditions

  61. [62]

    Avg. Score

    Enable Experience Harvesting Long-term effect Execution Traces quality success/failure latency network stats Profile Update / Learning Better Future Allocation Learns candidate quality Updates reliability & link profiles Improves future routing / hedge selection Redundancy is not only a protection mechanism for the current invocation, but also a data-coll...

    2024