Pith. sign in

REVIEW 2 major objections 6 minor 35 references

Noise Floor Audit for Agent Benchmarks

T0 review · 2 major / 6 minor · reviewed 2026-08-27 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read At temperature 0, rerunning the three audited tool-calling endpoints changes scores by less than 1.1 percentage points, while semantics-preserving prompt rewrites move matched scores by 5.2 to 21 percentage points—so leaderboard gaps…

desk verdict The rerun floor is real and well-measured, but the headline 11x-58x perturbation multiplier compares per-instance SDs to aggregate SDs; rescaled, the claim mostly collapses and reverses on one endpoint. read the letter →

arxiv 2608.22331 v1 pith:IQFBHFAQ submitted 2026-08-23 cs.CL

Yihang Chen , Pin Qian , Su Wang , Chong Peng , Huan Xu , Xiyang Wu , Yiqi Sun This is my paper · ORCID
classification cs.CL
keywords noisefloorauditfunctioncallingbenchmarkBFCLpromptperturbationpaireddifferencererundeterminismfailuretaxonomyLLMevaluationvariance
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish where measurement noise actually lives when scoring native tool-calling endpoints on the Berkeley Function Calling Leaderboard's AST-graded multiple and parallel categories. It reports that at temperature 0, reruns are nearly deterministic, with ever-flip fractions below 3% and rerun paired SDs at or below 1.1 percentage points. Semantics-preserving prompt perturbations produce a much larger floor: median paired SDs 11x to 58x larger than rerun paired SDs. If correct, leaderboard gaps on this suite should be compared against the prompt-perturbation floor, and marginal accuracy should be accompanied by failure-character reporting.

What carries the argument

The analysis rests on a matched paired-difference design: every endpoint, rerun, and prompt variant is scored on the same 150 instances with AST exactness, and variability is reported as the standard deviation of per-instance score differences rather than the dispersion of aggregate means. This paired quantity is the right target for a leaderboard gap A−B, and observed paired-to-√2-marginal ratios near unity confirm the usual independent-measurement approximation. The perturbation arm uses four named surface-form variants—whitespace compaction, a tool-instruction prefix, request-boundary labels, and a call-only suffix—that hold the schema, target function set, and ground truth fixed. A failure taxonomy splits incorrect outcomes into wrong function, wrong arguments, malformed output, and other, separating structural interface failures from semantic tool-use errors.

What would settle it

Inspect the perturbation arm instance-by-instance: if an independent check—human adjudication or a semantic entailment probe—finds a variant that alters the requested function or argument values on any of the 150 frozen instances, then perturbation SDs are not pure measurement noise; the ratio to rerun SD should be recomputed after removing those meaning-changed cases.

Watch

Extended reading notes

Core claim

On 150 frozen BFCL instances, the paper finds that reruns at temperature 0 are nearly deterministic across all three endpoints: ever-flip fractions are 0.7%, 2.0%, and 2.7%, with mean run correlations of 0.997, 0.966, and 0.961, and rerun paired SDs of 0.28pp, 0.91pp, and 1.1pp. The larger measurement floor comes from semantics-preserving prompt perturbations, whose median paired SDs are 16pp, 10pp, and 19pp—about 58x, 11x, and 16x the rerun floor. The perturbations mostly reshuffle which matched instances pass, rather than moving the endpoint mean by a large amount. The failure character also shifts across endpoints: malformed-output failures account for 30%, 7%, and under 1% of task failures, so the weaker endpoint fails structurally while the stronger ones fail with well-formed but wrong calls. Marginal accuracy hides these layers of measurement behavior.

Load-bearing premise

The load-bearing premise is that the four named prompt variants preserve the requested tool-call semantics on every instance; if any variant changes meaning for some tasks, the large perturbation SDs mix task change with measurement noise and the 11x–58x ratio overstates the prompt-surface floor.

Editorial extensions

If this is right

  • A leaderboard gap smaller than the paired perturbation SD (5.2–21 pp across these endpoints) cannot be distinguished from prompt-wording luck.
  • Extra reruns at temperature 0 buy almost nothing: after the first small rerun check, evaluation compute is better spent on prompt perturbations, grader audits, or broader instance coverage.
  • Marginal accuracy alone hides failure mode: malformed-output shares range from 30% to below 1% across endpoints, pointing to interface reliability on weaker endpoints versus semantic tool-use errors on stronger ones.
  • Benchmark maintainers who report only a single accuracy number should also publish the frozen instance list, the prompt-template family, paired perturbation SDs, and a small failure-character table.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The 11x–58x ratio is likely a lower bound on prompt-surface sensitivity in general, since the four variants form a narrow declarative family; richer wrappers, system messages, or few-shot examples could move scores even more.
  • A testable extension is to compute the paired perturbation SD for other tool-calling suites, such as JSON-only prompting or multi-turn agent tasks; if it stays this large, the reporting contract proposed here becomes a general benchmark requirement.
  • Because the perturbation floor dwarfs the rerun floor, leaderboard deltas should be accompanied by a matched-pair significance test, and the paired SDs reported here give the effect size needed to pre-register such budgets.
  • The reversal between rerun and perturbation stability across the two Groq sizes suggests stability is a per-endpoint property, not a monotone function of capability—worth checking whether a third size class or a temperature-0.7 arm reproduces the reversal.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Request a human review

A listed scientist reviews the paper for a fee and the review publishes here regardless of verdict. See the reviewers or get listed.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 6 minor

Summary. The paper reports a noise-floor audit of three native tool-calling endpoints (Llama 3.1 8B and Llama 3.3 70B on Groq, and a thinking-enabled Gemini 3.5 Flash) on the BFCL 'multiple' and 'parallel' categories. Using a frozen 150-instance matched set, the authors measure rerun variability at temperature 0 (N=10) and prompt-perturbation variability under four declared semantics-preserving surface edits (N=5). They report near-deterministic reruns (ever-flip fractions 0.7-2.7%, paired SD 0.28-1.1pp) and claim that prompt perturbations create a much larger floor (median paired SD 11-58x larger). They also provide a failure taxonomy showing that malformed-output failures account for 30%, 7%, and <1% of failures for the three endpoints and that marginal accuracy hides both stability and failure character.

Significance. If the claims held, the paper would be a valuable practical guide for leaderboard comparison: it demonstrates a careful matched-pair design, retains raw rows, separates instance-mix variance from repeated-measurement variance, and complements aggregate scores with a failure taxonomy. The rerun-floor measurements are directly useful, and the failure taxonomy is a meaningful contribution. The perturbation-sensitivity result, once recomputed at a consistent aggregation level, still indicates that prompt template choice can affect matched scores, though with markedly smaller effect sizes than advertised. The paper's methodological contribution—making uncertainty visible at the level at which benchmark claims are made—is appropriate for the journal.

major comments (2)
  1. [Section 4, Tables 2-3, and Appendix A.2] The headline 11x-58x multipliers compare incompatible statistics. Table 2's 'Paired SD' is the SD of aggregate-score differences across rerun pairs (0.28pp, 0.91pp, 1.1pp), as confirmed by the paired-to-sqrt(2)-marginal ratios in Section 3. Table 3's 'Matched Paired SD' (0.096-0.209) is the SD of per-instance binary differences across 750 matched instance-rerun pairs; the Appendix A.2 caption explicitly warns that such instance-level SDs are not comparable in magnitude to Table 2's aggregate SDs. Dividing the per-instance perturbation SD by the aggregate rerun SD yields the reported multiples, but the correct aggregate perturbation SD for a 150-instance evaluation is the per-instance SD divided by sqrt(150). Recomputing gives approximately 4.7x (8B), 0.9x (70B), and 1.4x (Gemini); the claim that perturbations are the larger floor on all endpoints does not survive for 70B and is marginal for Gemini. The abstract, contributions, and Discussion must be revised to use a consistent aggregation level.
  2. [Section 3, Table 1] The four prompt variants are asserted to preserve the requested tool-call semantics, but no independent verification is provided (e.g., human annotation, LLM-based equivalence checking, or measurement of downstream task outcomes). The 'tool instruction prefix' and 'call only suffix' variants in particular could change a model's decision to emit a call or its output format for some instances. If any variant alters the task for a subset of instances, the perturbation paired SD is not a pure measurement floor but also reflects task change, and the 'noise floor' framing is weakened. Please provide evidence of semantic preservation or reframe the result as 'prompt-surface sensitivity' rather than 'noise floor.'
minor comments (6)
  1. [Section 3, 'Reported uncertainty'] Define explicitly whether the 'paired quantity' is at the aggregate level or the per-instance level, and use distinct terms (e.g., 'aggregate paired SD' vs 'instance-level paired SD') throughout to avoid the confusion that led to the multiplier error.
  2. [Table 3 caption] Add a note that the 'Matched Paired SD' is at the instance level (comparable to the SD columns in Table A.2), not at the aggregate level of Table 2.
  3. [Figure 1] After correcting the aggregation, update the log-scale comparison so that the two plotted quantities are at the same level; consider plotting the aggregate perturbation SD instead of the instance-level median.
  4. [Abstract and Section 1] Qualify 'semantics-preserving' as 'declared semantics-preserving' given the lack of verification, and adjust the '11x to 58x' claim to the corrected values.
  5. [Section 4, 'Bootstrap sample-size curve'] The text states endpoint-specific means of 0.26pp, 1.1pp, and 1.3pp; ensure the endpoint ordering matches the order in Table A.1 for clarity.
  6. [General] The paper emphasizes auditability and raw row retention; a data/code availability statement would help readers reproduce the matched-pair analyses.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: direct measurement audit against external BFCL suite and AST grader; the 11x–58x ratio concern is a comparability issue, not a derivation from inputs.

full rationale

The paper's claims are direct empirical measurements, not derived quantities that reduce to their own inputs. Rerun paired SDs, perturbation paired SDs, ever-flip fractions, and failure-taxonomy shares are all computed from retained raw outputs against the external BFCL benchmark and a programmatic AST grader. No parameter is fitted to one subset and then renamed as a prediction; no quantity is defined in terms of the result it is used to establish. The semantics-preserving label on the four prompt perturbations is a stated design assumption, not a conclusion forced by the measurement. The skeptical concern that Table 2's aggregate paired SDs are compared with Table 3's per-instance matched SDs is a statistical-comparability objection, not a circularity objection: an apples-to-oranges ratio is an error in interpretation, but the ratio is not made true by construction and does not rely on self-citation. There are no load-bearing self-citations; the cited works are external related work and the official BFCL dataset. Therefore the derivation chain is self-contained with respect to the audit's inputs, and the circularity score is 0.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

No fitted parameters are present; all reported quantities are descriptive statistics. The analysis rests on three stated domain assumptions: semantic preservation of the perturbations, validity of the AST grader, and adequacy of the frozen matched set. No new entities are introduced.

assumptions (3)
  • ad hoc to paper The four named prompt variants preserve the requested tool-call semantics.
    Table 1 asserts each variant changes only user-message surface form while keeping schema, function set, and ground truth fixed; this premise is what lets the perturbation SD be read as measurement noise rather than task change.
  • domain assumption The official BFCL AST grader correctly classifies tool-call correctness for the multiple and parallel categories.
    All scores and the failure taxonomy inherit the grader boundary; the paper itself notes that programmatic matchers can miss semantically correct calls and over-credit superficial matches (Section 6).
  • domain assumption The frozen 150-instance study set is an adequate matched slice of the two BFCL categories for the audit's conditional claims.
    The paper uses a hash-frozen set and removes instance-mix variance by design, but does not sample from the category population; generalizations beyond this set are unsupported, as stated in Section 6.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Noise Floor Audit for Agent Benchmarks." pith.science (2026). https://pith.science/paper/IQFBHFAQ

@misc{pith2026260822331,
  author       = {Pith},
  title        = {Pith review of: Noise Floor Audit for Agent Benchmarks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/IQFBHFAQ}},
  note         = {Machine review of arXiv:2608.22331}
}
read the original abstract

We audit measurement variability for 3 native tool-calling endpoints across 2 providers on the official BFCL multiple and parallel categories, using matched AST grading. At temperature 0, reruns are nearly deterministic across Groq endpoints and a thinking-enabled Gemini setting: ever-flip fractions are 0.7%, 2.0%, and 2.7%, with mean run correlations of 0.997, 0.966, and 0.961. Semantics-preserving prompt perturbations create the larger floor on all endpoints, with median perturbation paired SDs 11x to 58x larger than rerun paired SDs. The failure character also shifts: malformed-output failures account for 30%, 7%, and <1% of task failures, so marginal accuracy hides not only stability but also failure mode.

Figures

Figures reproduced from arXiv: 2608.22331 by the authors.

Figure 1
Figure 1. Rerun paired SD versus median perturbation paired SD for each endpoint. [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

35 extracted references · 34 canonical work pages

  1. [1]

    2024 , eprint =

    Yao, Shunyu and Shinn, Noah and Razavi, Pedram and Narasimhan, Karthik , journal =. 2024 , eprint =

    work page 2024

  2. [2]

    Adding Error Bars to Evals: A Statistical Approach to Language Model Evaluations

    Adding Error Bars to Evals: A Statistical Approach to Language Model Evaluations , author =. arXiv preprint arXiv:2411.00640 , year =. 2411.00640 , archivePrefix =

    work page Pith review arXiv

  3. [3]

    Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How

    Sclar, Melanie and Choi, Yejin and Tsvetkov, Yulia and Suhr, Alane , booktitle =. Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How. 2024 , eprint =

    work page 2024

  4. [4]

    State of What Art? A Call for Multi-Prompt LLM Evaluation

    Mizrahi, Moran and Kaplan, Guy and Malkin, Dan and Dror, Rotem and Shahaf, Dafna and Stanovsky, Gabriel , journal =. State of What Art? A Call for Multi-Prompt. 2024 , doi =. 2401.00595 , archivePrefix =

    work page Pith review arXiv 2024

  5. [5]

    PromptRobust: Towards Evaluating the Robustness of Large Language Models on Adversarial Prompts

    Zhu, Kaijie and Wang, Jindong and Zhou, Jiaheng and Wang, Zichen and Chen, Hao and Wang, Yidong and Yang, Linyi and Ye, Wei and Zhang, Yue and Gong, Neil Zhenqiang and Xie, Xing , year =. doi:10.48550/arXiv.2306.04528 , url =. 2306.04528 , archivePrefix =

    work page Pith review arXiv doi:10.48550/arxiv.2306.04528

  6. [6]

    What Did I Do Wrong? Quantifying LLM s' Sensitivity and Consistency to Prompt Engineering

    Errica, Federico and Sanvito, Davide and Siracusano, Giuseppe and Bifulco, Roberto , booktitle =. What Did. 2025 , publisher =. doi:10.18653/v1/2025.naacl-long.73 , url =

    work page doi:10.18653/v1/2025.naacl-long.73 2025

  7. [7]

    Same Question, Different Answer? Measuring and Mitigating Prompt Privilege for Equitable AI Access

    Jin, Lier and Hu, Lan and Shen, Binqi and Cai, Hanyu and Xin, Yuting , journal =. Same Question, Different Answer? Measuring and Mitigating Prompt Privilege for Equitable. 2026 , eprint =. doi:10.48550/arXiv.2608.08942 , url =

    work page Pith review arXiv doi:10.48550/arxiv.2608.08942 2026

  8. [8]

    OLMES : A standard for language model evaluations

    Gu, Yuling and Tafjord, Oyvind and Kuehl, Bailey and Haddad, Dany and Dodge, Jesse and Hajishirzi, Hannaneh , booktitle =. 2025 , publisher =. doi:10.18653/v1/2025.findings-naacl.282 , url =

    work page doi:10.18653/v1/2025.findings-naacl.282 2025

Show all 35 references
  1. [9]

    Advances in Neural Information Processing Systems , volume =

    Signal and Noise: A Framework for Reducing Uncertainty in Language Model Evaluation , author =. Advances in Neural Information Processing Systems , volume =. 2025 , doi =

    2025

  2. [10]

    doi:10.48550/arXiv.2605.25492 , url =

    Li, Yanhang and Fan, Zhichao and Zhuang, Zexin , year =. doi:10.48550/arXiv.2605.25492 , url =. 2605.25492 , archivePrefix =

    doi arXiv

  3. [11]

    Pre-Registering the Detectable Effect: A Paired-

    Zhuang, Zexin and Li, Yanhang and Fan, Zhichao , year =. Pre-Registering the Detectable Effect: A Paired-. doi:10.48550/arXiv.2605.28873 , url =. 2605.28873 , archivePrefix =

    doi arXiv

  4. [12]

    arXiv preprint arXiv:2405.14782 , year =

    Lessons from the Trenches on Reproducible Evaluation of Language Models , author =. arXiv preprint arXiv:2405.14782 , year =. 2405.14782 , archivePrefix =

    arXiv

  5. [13]

    Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics , year =

    The Hitchhiker's Guide to Testing Statistical Significance in Natural Language Processing , author =. Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics , year =

  6. [14]

    Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing , year =

    Show Your Work: Improved Reporting of Experimental Results , author =. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing , year =

    2019

  7. [15]

    2024 , publisher =

    Chen, Zehui and Du, Weihua and Zhang, Wenwei and Liu, Kuikun and Liu, Jiangning and Zheng, Miao and Zhuo, Jingming and Zhang, Songyang and Lin, Dahua and Chen, Kai and Zhao, Feng , booktitle =. 2024 , publisher =. doi:10.18653/v1/2024.acl-long.515 , url =

    2024 doi

  8. [16]

    doi:10.48550/arXiv.2411.13547 , url =

    Kokane, Shirley and Zhu, Ming and Awalgaonkar, Tulika and Zhang, Jianguo and Hoang, Thai and Prabhakar, Akshara and Liu, Zuxin and Lan, Tian and Yang, Liangwei and Tan, Juntao and Murthy, Rithesh and Yao, Weiran and Liu, Zhiwei and Niebles, Juan Carlos and Wang, Huan and Heine...

    doi arXiv

  9. [17]

    , year =

    Sun, Xian and Gao, Wei and Wang, Yingshuo and Kong, Lingdong and Li, Yanhang and Fan, Zhichao and Zhuang, Zexin and Dong, Wenlong and Zheng, Zhiyuan and Paranjape, Hrishikesh and Mandal, Abhishek and Zhang, Johnny R. , year =. Beyond Accuracy: Measuring Bias Acknowledgment in ...

    doi arXiv

  10. [18]

    2026 , eprint =

    Do Time Series Foundation Model Benchmarks Hide Regime-Dependent Failures? Evidence from Traffic Speed Forecasting , author =. 2026 , eprint =. doi:10.48550/arXiv.2606.18367 , url =

    2026 doi arXiv

  11. [19]

    Ma, Chang and Zhang, Junlei and Zhu, Zhihao and Yang, Cheng and Yang, Yujiu and Jin, Yaohui and Lan, Zhenzhong and Kong, Lingpeng and He, Junxian , booktitle =

  12. [20]

    Learning Human-Perceived Fakeness in

    Fu, Xingyu and Liu, Siyi and Xu, Yinuo and Lu, Pan and Hu, Guangqiuse and Yang, Tianbo and Anantasagar, Taran and Shen, Christopher and Mao, Yikai and Liu, Yuanzhe and Shah, Keyush and Lee, Chung Un and Choi, Yejin and Zou, James and Roth, Dan and Callison-Burch, Chris , journ...

    2025 doi

  13. [21]

    and Zhang, Tianjun and Wang, Xin and Gonzalez, Joseph E

    Patil, Shishir G. and Zhang, Tianjun and Wang, Xin and Gonzalez, Joseph E. , booktitle =. Gorilla: Large Language Model Connected with Massive. 2024 , eprint =

    2024

  14. [22]

    2023 , eprint =

    Qin, Yujia and Liang, Shihao and Ye, Yining and Zhu, Kunlun and Yan, Lan and Lu, Yaxi and Lin, Yankai and Cong, Xin and Tang, Xiangru and Qian, Bill and Zhao, Sihan and Hong, Lauren and Tian, Runchu and Xie, Ruobing and Zhou, Jie and Gerstein, Mark and Li, Dahai and Liu, Zhiyu...

    2023

  15. [23]

    2026 , eprint =

    Deng, Ryan and Liu, Yuanzhe and Lipka, Bastian and Ma, Yao and Chen, Xuhao and Kaler, Tim and Ganhotra, Jatin , journal =. 2026 , eprint =. doi:10.48550/arXiv.2607.19653 , url =

    2026 doi arXiv

  16. [24]

    and Ma, Yao and Chen, Jie , journal =

    Liu, Yuanzhe and Deng, Ryan and Kaler, Tim and Chen, Xuhao and Leiserson, Charles E. and Ma, Yao and Chen, Jie , journal =. Lessons Learned: A Multi-Agent Framework for Code. 2025 , eprint =. doi:10.48550/arXiv.2505.23946 , url =

    2025 doi

  17. [25]

    From Agent Traces to Trust: A Survey of Evidence Tracing and Execution Provenance in

    Wang, Yiqi and Zhang, Jiaqi and Wu, Zhangkai and Cai, Taotao and Liu, Zirui and Sun, Qingqiang and Sun, Zequn and Dong, Manqing and Zheng, Mingkai and Yin, Xuefei and Zhu, Yanming , journal =. From Agent Traces to Trust: A Survey of Evidence Tracing and Execution Provenance in...

    2026 doi arXiv

  18. [26]

    2026 , eprint =

    Li, Xiaoyang and Wang, Yiqi and Lu, Haohui and Chen, Zhi and Li, Mo and Song, Pingan and Zheng, Mingkai and Cai, Taotao , journal =. 2026 , eprint =. doi:10.48550/arXiv.2607.23929 , url =

    2026 doi arXiv

  19. [27]

    arXiv preprint arXiv:2608.10502 , year =

    From Faulty Memories to Corrected Actions: Dependency-Guided Rollback Repair for Memory-Augmented Agents , author =. arXiv preprint arXiv:2608.10502 , year =. doi:10.48550/arXiv.2608.10502 , url =. 2608.10502 , archivePrefix =

    doi arXiv

  20. [28]

    2026 , eprint =

    Wang, Yiqi and Yan, Zihao and Zhang, Jiaqi and Wu, Zhangkai and Zheng, Mingkai and Sun, Zequn and Zhu, Yanming and Cai, Taotao , journal =. 2026 , eprint =. doi:10.48550/arXiv.2608.10509 , url =

    2026 doi arXiv

  21. [29]

    arXiv preprint arXiv:2605.11951 , year =

    From Reaction to Anticipation: Proactive Failure Recovery through Agentic Task Graph for Robotic Manipulation , author =. arXiv preprint arXiv:2605.11951 , year =. doi:10.48550/arXiv.2605.11951 , url =. 2605.11951 , archivePrefix =

    doi arXiv

  22. [30]

    2026 , eprint =

    How Early Is Early Enough? Design-Dependent Observation-Window Sufficiency in Subscription Churn Prediction , author =. 2026 , eprint =. doi:10.48550/arXiv.2607.00473 , url =

    2026 doi arXiv

  23. [31]

    doi:10.48550/arXiv.2603.01168 , url =

    Fu, Rong and Meng, Chunlei and Liu, Jinshuo and Zhao, Dianyu and Liu, Yongtai and Meng, Yibo and Ma, Xiaowen and Wu, Wangyu and Zeng, Yangchen and Cao, Shuaishuai and Fong, Simon , year =. doi:10.48550/arXiv.2603.01168 , url =. 2603.01168 , archivePrefix =

    doi arXiv

  24. [32]

    2026 , eprint =

    Chain-Aware Encoding for Microservice Trace Anomaly Detection , author =. 2026 , eprint =. doi:10.48550/arXiv.2607.10156 , url =

    2026 doi arXiv

  25. [33]

    Agentic-

    Su, Yiyun and Zhu, Huiying and Tian, Yu and Zhao, Changruo and Peng, Zujun and Liu, Yuting and Zhang, Luyan and Fan, Liang and Li, Baihua , editor =. Agentic-. Advanced Intelligent Computing Technology and Applications , series =. 2027 , publisher =. doi:10.1007/978-981-92-343...

    2027 doi

  26. [34]

    2026 , howpublished =

    Berkeley Function-Calling Leaderboard , author =. 2026 , howpublished =

    2026

  27. [35]

    2026 , howpublished =

    Berkeley Function Calling Leaderboard Dataset , author =. 2026 , howpublished =

    2026

Pith tools

Reviewed August 27, 2026 · model on record in the stance chip above.