Pith. sign in

REVIEW 3 major objections 5 minor 62 references

UrbanAgent: A Tool-Augmented Agent for Cross-System Urban Tasks

T0 review · 3 major / 5 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A tool-augmented agent that asks for missing inputs before acting completes 71% of cross-system urban requests, ten points above the strongest general baseline under the same model and tool budget.

desk verdict UrbanEval's process-layer metrics are the real contribution, and the 10-point TSR claim is plausible but rests on an unvalidated LLM judge plus no released artifacts. read the letter →

arxiv 2608.03018 v1 pith:QYSPYWRM submitted 2026-08-04 cs.AI

Jiayu Cao , Xingyuan Zeng , feiyu Li , Zhijing Huang , Xujie Yuan , Rongxiang Chen , Shimin Di , Libin Zheng
show 1 more author
Jian Yin
This is my paper · ORCID
classification cs.AI
keywords tool-augmentedagentscross-systemurbantasksAgentEvallargelanguagemodelsModelContextProtocolclarificationbeforeactingevidencegrounding
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that a single adaptive loop—ask for missing task-defining inputs before touching any tool, ground each action in the live observation it returns, and refuse to state facts the trace does not support—lets one agent finish complex cross-system city requests substantially more often than generic tool orchestration. It introduces UrbanEval, a 250-request benchmark spanning five task shapes and three difficulty levels, scored both on the answer and on the execution trace. In the controlled comparison with GPT-5-mini, UrbanAgent reaches 71% task success over all queries and 64.0% over the 200 executable ones, against 61% and 53.5% for the strongest general baselines under the same tools and budget. The result matters because real urban requests combine services—weather, routing, restaurant rules—that current platforms handle in isolation, and the paper's analysis finds the gain is not model scale or tool selection but disciplined verification and proactive augmentation.

What carries the argument

The load-bearing object is the structured task record $U=\langle I,E,C_{\mathrm{hard}},C_{\mathrm{soft}},M\rangle$ produced by the cognition component: intent $I$, entities and parameters $E$, mandatory constraints $C_{\mathrm{hard}}$, soft preferences $C_{\mathrm{soft}}$, and the set $M$ of missing user-specific inputs that cannot be inferred without changing the task. When $M$ is non-empty the loop halts and asks the user before any tool call. Otherwise the reasoning–execution core alternates action and observation so that downstream calls consume upstream values; the runtime rejects empty or region-inconsistent geocodes, caches only usable call signatures, and switches tools after eight consecutive failures; and the synthesis step cleans the draft, aligns it to the most relevant observation, and verifies it against $C_{\mathrm{hard}}$, admitting no fact absent from the observation history. This loop, not the underlying model, is what carries the reported completion gain.

What would settle it

Re-run the 250 queries twice: once with every system receiving identical cached tool responses, and once with human raters blind to system identity scoring the same TSR/MIR/VPAR rubric; if the human-scored gap between UrbanAgent and the strongest baseline drops well below the reported ten points, the headline gain is an artifact of the judge or of service timing.

Watch

Extended reading notes

Core claim

The paper's central claim is empirical: cross-system urban task completion is gated less by reasoning power than by workflow discipline. UrbanAgent structures execution as a closed loop—clarify, reason, act, observe, verify—so that location and time are never silently assumed, downstream calls consume upstream values, syntactically valid but geographically inconsistent returns are rejected, and the final answer contains no fact absent from the observation history. On UrbanEval this structure completes 71% of all 250 queries and 64.0% of the 200 executable ones, against 61% overall and 53.5% on executable queries for the strongest general baselines under matched conditions. The analysis localizes the gap: tool coverage, invocation order, and clarification rates are nearly saturated across systems, while only proactive augmentation (0.96 vs. 0.84) and task completeness separate UrbanAgent from the pack, with the failure count on incomplete tasks dropping from 31–45 for baselines to 20 for UrbanAgent.

Load-bearing premise

The whole comparison rests on the Qwen3-Max LLM judge scoring success, missing-input flags, and proactive augmentation the way a human would, with no reported human agreement or calibration, so if that judge systemically favors UrbanAgent's structured outputs, or if per-run service timing biases the comparisons, the ten-point gap collapses.

Editorial extensions

If this is right

  • If the claim is right, a clarification-first, evidence-aligned loop is sufficient to convert high-level city requests into executable workflows far more often than generic orchestration: on GPT-5-mini, task success rises from 22% with a tool-free model to 71% with UrbanAgent, while the best tool-free models stay within 22–26% regardless of scale.
  • The gain is not evenly spread: it is largest on factual retrieval (+16 points) and conditional or combinatorial decision-making (+12 points), smallest on sequential dependency where most systems already reach 0.68–0.72, and it reverses on complex retrieval across independent services where a multi-agent conversation baseline reaches 0.50 vs. 0.48.
  • The advantage appears across four tested base models, though the margin shrinks to +2 points on DeepSeek-V4-flash where the strongest baseline already reaches 0.50 task success, so the framework helps most where baselines are weakest.
  • Reliability costs tokens: per successful task UrbanAgent uses roughly three times the tokens of the cheapest baselines, so a practical user would trade cost for completion only on requests where silent assumptions are dangerous.
  • Execution-trace metrics separate genuine completion from fluent answers: since tool coverage and invocation order are nearly equal across systems, a benchmark that scores only the final answer would miss most of the gap between UrbanAgent and its competitors.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: the same clarify-then-verify loop should transfer to other multi-system service domains—travel booking, insurance claims, healthcare referral chains—where a high-level request hides missing user-specific inputs and a plausible answer can be built without ever touching the right service; the paper only demonstrates the loop in seven urban-computing domains.
  • Beyond the paper: because the benchmark grades time-varying facts against each run's own tool returns, the reported gap is not yet shown to be service-independent; a controlled rerun with cached, identical tool responses for all systems would tell whether the ten points come from orchestration skill or from which values the live services happened to return.
  • Beyond the paper: the near-flat 22–26% task success across ten tool-free models is consistent with the hypothesis that live verifiable observations, not parametric knowledge, is the binding constraint for open-world service tasks; an experiment that feeds a fixed service snapshot to both tool-free and tool-augmented models would quantify how much of the tool gain is data freshness rather than tool
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes UrbanAgent, a tool-augmented LLM agent framework for cross-system urban tasks, and introduces UrbanEval, a 250-query benchmark spanning five task categories and three difficulty levels. UrbanAgent couples a cognition module that clarifies missing task-critical inputs before acting, a reasoning–execution core that selects actions incrementally, a tool-invocation layer with geographic and empty-result validation, and an evidence-aligned synthesis stage that admits no fact absent from the collected observations. The experiments compare UrbanAgent against Native tool-calling, ReAct, Plan-and-Execute, and AutoGen under matched tools, schemas, temperature, and call budget across four base models, plus ten single LLMs without tools. The headline result is an overall TSR of 71% versus 61% for the strongest baseline, and 64.0% versus 53.5% on the 200 executable queries with GPT-5-mini. The authors argue the gain is localized to proactive augmentation and task completeness rather than tool coverage, ordering, or clarification, and that the advantage persists across all tested base models.

Significance. If the central claim is correct, the paper makes a useful contribution: it demonstrates that a clarification-first, evidence-aligned orchestration design can convert complex natural-language urban requests into executable cross-system workflows substantially more often than standard agent frameworks, under carefully matched tool access. The experimental design has real strengths: tools, schemas, temperature, and per-query budgets are matched across baselines; TIC and IOA are computed deterministically from traces; the failure analysis separates incompleteness from factual error and requirement mismatch; and the evaluation across four base models is a genuine robustness check. The authors also mitigate the self-evaluation concern by freezing queries and schema before finalizing UrbanAgent, annotating the schema independently, and masking judge identities. However, the central claim rests on an unvalidated LLM judge, and the evaluation protocol measures time-varying facts against each run's own tool returns rather than an external key.

major comments (3)
  1. [Evaluation (Table 1)] The headline TSR, MIR, and VPAR numbers are produced by a single LLM judge, Qwen3-Max at temperature 0, and no human agreement, inter-judge reliability, or calibration is reported. This is load-bearing because the rubric's rule that a value 'is only recognized when the trajectory actually generates the input behind it' is exactly the mechanism implemented by UrbanAgent's synthesis component, which 'admits no fact absent from O and R'. A baseline that produces the correct answer without explicit provenance may therefore be penalized while UrbanAgent is rewarded for formatting, not necessarily for task completion. Please report human agreement on a random subset, at least a second independent judge, and an ablation that rewrites or strips final answers into a common format before judging; without this, the 10-point gap in Table 2 is not yet established.
  2. [Evaluation (time-varying facts)] The schema grades time-varying facts against each run's own tool returns rather than a fixed external key. This makes TSR a measure of self-consistent transcription from the trajectory, so a run that faithfully reports a stale or wrong service value is credited as successful. The paper's claim is about completing real cross-system urban tasks, not about internal consistency; please add spot checks of time-varying values against an independent ground-truth source or a re-run with fixed service snapshots, and report the agreement rate. The limitations paragraph acknowledges that evaluation 'does not establish generalization to unseen cities or domains,' but it does not address this internal-validity issue.
  3. [Experiments (Controlled comparison, Table 2)] All reported TSR values come from a single run per query at temperature T=1.0, with no seeds, repetitions, confidence intervals, or significance tests. The headline gap is 10 points on 250 queries, which is on the order of two standard errors, and the robustness margin varies from +2 (DeepSeek-V4-flash) to +14 (Gemini-2.5-flash). Please report bootstrap intervals, multiple seeds, or paired significance tests before claiming a stable rank across base models; the +2 margin on DeepSeek-V4-flash is compatible with no advantage.
minor comments (5)
  1. [UrbanEval Benchmark] The sentence 'Factsthatchangeover time such as current weather are compared with the raw tool outputs from' is incomplete; the trailing 'from' appears to be a typo.
  2. [Table 2] The 'Tok.↓' column does not specify its unit; the text refers to 'k' tokens, so the header or caption should state that values are in thousands of tokens.
  3. [Abstract and Introduction] The paper uses both 'UrbanAgent' and 'Urban-Agent'; please standardize the spelling throughout.
  4. [Main Results] The text says 'the strongest baselines, Native tool-calling and AutoGen' but later uses 'the strongest baseline' in the singular; clarify which baseline is used for each comparison.
  5. [Analysis (Q2)] The phrase 'There are two behaviors can explain' is grammatically incomplete and should read 'There are two behaviors that can explain'.

Circularity Check

1 steps flagged · score 6.0 of 10

UrbanAgent's headline TSR advantage is partially produced by an evaluation rubric whose evidence-recognition rule is the same provenance constraint UrbanAgent's synthesis is engineered to satisfy.

  1. self definitional [UrbanEval Benchmark, schema/criteria paragraph; UrbanAgent, Evidence-Aligned Synthesis; Experiments, Evaluation]
    "Therefore, the value is only recognized when the trajectory actually generates the input behind it."

    UrbanEval scores TSR for time-varying and computed values by checking that the final answer is backed by the run's own tool outputs, because "the schema grades time-varying facts against each run's own returns rather than a fixed key." UrbanAgent's synthesis applies the same relation as a design constraint: "Verify checks the conclusion against the constraints in U and admits no fact absent from O and R." A final response that merely restates the run's observations is therefore credited as task success even when the underlying value is stale or wrong relative to external ground truth, and UrbanAgent is constructed to produce exactly this evidence-restatement property.

full rationale

The circularity is partial, not total. The core comparison is run under matched models, tools, budgets, and queries, and the UrbanEval queries and schema were frozen before UrbanAgent was finalized, so the relative design of the agent is not itself circular. Deterministic process metrics (TIC, IOA) and fixed-fact TSR checks against fixed references retain independent content. However, the metric that carries the headline finding (TSR on time-varying and tool-derived facts) uses an evidence-traceability rule that is logically identical to UrbanAgent's Verify-and-Align synthesis constraint, and the LLM judge used to apply that rule is never calibrated against human labels. The paper's own limitation sentence concedes that "evaluation with time-varying services and an LLM judge does not establish generalization." No load-bearing self-citation chain was found; the only overlapping-author citation (Code2MCP, Ouyang et al. 2025) is an implementation detail for MCP tooling, not evidence for the central claim.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

No fitted scientific parameters are present; the listed free parameters are hand-chosen engineering constants. The central claims rest on the validity of the LLM judge, the self-built benchmark, and the tool-return ground-truth assumption. No new physical or conceptual entities are postulated, so the invented-entities ledger is empty.

free parameters (4)
  • Per-query LLM call budget = 25 calls or 600 seconds
    Hand-chosen cap on agent computation; matched across all systems for fairness, but it constrains which tasks are completable and influences relative rankings.
  • Sampling temperature for agent policy = 1.0
    Hand-chosen for the controlled GPT-5-mini setting; token counts and success rates could shift at other temperatures.
  • Consecutive-failure retry threshold = 8
    Hand-chosen cutoff after which the runtime advises switching tools, impacting error recovery and final success.
  • Judge temperature = 0
    Hand-chosen for Qwen3-Max; score distributions from the judge are not validated against human raters.
assumptions (4)
  • domain assumption Qwen3-Max, masked and at temperature 0, produces valid TSR, MIR, and VPAR scores.
    No human agreement, calibration, or error analysis is reported; the headline TSR gap rests on these scores (Evaluation section).
  • domain assumption Each run's own tool returns are the correct ground truth for time-varying facts.
    The schema grades time-varying facts against each run's own returns, so a service returning stale or incorrect data is scored as correct as long as the agent echoes it (Evaluation section).
  • domain assumption The 250 hand-written UrbanEval queries and seven MCP domains are representative of real cross-system urban requests.
    The benchmark is self-constructed and not externally validated; generalization to unseen cities or domains is explicitly unproven (Limitations).
  • ad hoc to paper The geographic-consistency and empty-result checks detect all unusable tool returns.
    These are project-specific heuristics that classify observation validity; errors here propagate into evidence grounding (Tool Invocation and Grounding section).

how reviews work

0 comments
Cite this review

Pith. "Pith review of UrbanAgent: A Tool-Augmented Agent for Cross-System Urban Tasks." pith.science (2026). https://pith.science/paper/QYSPYWRM

@misc{pith2026260803018,
  author       = {Pith},
  title        = {Pith review of: UrbanAgent: A Tool-Augmented Agent for Cross-System Urban Tasks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QYSPYWRM}},
  note         = {Machine review of arXiv:2608.03018}
}
read the original abstract

Modern cities rely on an increasing number of digital services to operate, but residents' daily needs are still difficult to meet. Services are fragmented and have little interoperability, placing a heavy operational burden on users. Existing digital platforms, urban foundation models, and intelligent assistants each address only isolated aspects of an urban task. But they struggle to reliably convert complex natural-language requests into executable cross-system workflows. We propose Urban-Agent, a tool-augmented agent framework for cross-system urban tasks. It couples the cognitive and reasoning capabilities of a large language model with a tool-set supporting code execution, API calls, and Model Context Protocol. Through one adaptive closed loop, it clarifies missing information before acting, grounds tool use in live observations, and aligns the final response with observed evidence and task constraints. To address the evaluation gap, we introduce Urban-Eval, a benchmark specifically designed for cross-system urban request. Unlike prior benchmarks that assess either general tool use or urban knowledge and reasoning, Urban-Eval evaluates both task results and execution quality, including required tool coverage, dependency validity, and evidence traceability. Experimental results indicate that Urban-Agent reaches a 71% task success rate, 10 points above the strongest baseline. This lead holds across GPT-5-mini, Gemini-2.5-flash, DeepSeek-V4-flash, and Qwen3-235B-A22B.

Figures

Figures reproduced from arXiv: 2608.03018 by the authors.

Figure 1
Figure 1. Fragmented urban services vs UrbanAgent. (a)Existing approaches each cover one fragment of a compositional request. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Framework of UrbanAgent. (1) Cognition maps a query to a structured task [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Overview of UrbanEval. (a) The 250 queries span five task categories and three difficulty levels, and every category is [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Main results and analysis on GPT-5-mini. (a) TSR by query type: factual retrieval (FR), complex retrieval across [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

62 extracted references · 22 canonical work pages

  1. [1]

    ACM Transactions on Intelligent Systems and Technology (TIST) , volume=

    Urban computing: concepts, methodologies, and applications , author=. ACM Transactions on Intelligent Systems and Technology (TIST) , volume=. 2014 , publisher=

    work page 2014

  2. [2]

    ACM Transactions on Intelligent Systems and Technology , volume=

    Urban computing in the era of large language models , author=. ACM Transactions on Intelligent Systems and Technology , volume=. 2025 , publisher=

    work page 2025

  3. [3]

    Npj urban sustainability , volume=

    Smart cities and the urban digital divide , author=. Npj urban sustainability , volume=. 2023 , publisher=

    work page 2023

  4. [4]

    npj Urban Sustainability , volume=

    Urban-rural digitalization evolves from divide to inclusion: empirical evidence from China , author=. npj Urban Sustainability , volume=. 2024 , publisher=

    work page 2024

  5. [5]

    Yao, Shunyu and Zhao, Jeffrey and Yu, Dian and Du, Nan and Shafran, Izhak and Narasimhan, Karthik R and Cao, Yuan , booktitle=

  6. [6]

    Advances in neural information processing systems , volume=

    Toolformer: Language models can teach themselves to use tools , author=. Advances in neural information processing systems , volume=

  7. [7]

    Advances in Neural Information Processing Systems , volume=

    Gorilla: Large language model connected with massive apis , author=. Advances in Neural Information Processing Systems , volume=

  8. [8]

    2402.01680 , archivePrefix=

    Large language model based multi-agents: A survey of progress and challenges , author=. 2402.01680 , archivePrefix=

    arXiv

Show all 62 references
  1. [9]

    The Twelfth International Conference on Learning Representations , year=

    Gaia: a benchmark for general ai assistants , author=. The Twelfth International Conference on Learning Representations , year=

  2. [10]

    Zhou, Shuyan and Xu, Frank F and Zhu, Hao and Zhou, Xuhui and Lo, Robert and Sridhar, Abishek and Cheng, Xianyi and Ou, Tianyue and Bisk, Yonatan and Fried, Daniel and others , booktitle=

  3. [11]

    Xie, Tianbao and Zhang, Danyang and Chen, Jixuan and Li, Xiaochuan and Zhao, Siheng and Cao, Ruisheng and Hua, Toh J and Cheng, Zhoujun and Shin, Dongchan and Lei, Fangyu and others , journal=

  4. [12]

    Forty-second International Conference on Machine Learning , year=

    The berkeley function calling leaderboard (bfcl): From tool use to agentic evaluation of large language models , author=. Forty-second International Conference on Machine Learning , year=

  5. [13]

    Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V

    Evaluation and benchmarking of llm agents: A survey , author=. Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V. 2 , pages=

  6. [14]

    Introducing the Model Context Protocol , year =

  7. [15]

    Model Context Protocol Specification , year =

  8. [16]

    Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V

    Citybench: Evaluating the capabilities of large language models for urban tasks , author=. Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V. 2 , pages=

  9. [17]

    Zhou, Baichuan and Yang, Haote and Chen, Dairong and Ye, Junyan and Bai, Tianyi and Yu, Jinhua and Zhang, Songyang and Lin, Dahua and He, Conghui and Li, Weijia , booktitle=

  10. [18]

    2509.05941 , archivePrefix=

    Code2MCP: Transforming Code Repositories into MCP Services , author=. 2509.05941 , archivePrefix=

  11. [19]

    International Journal of Nursing Studies , volume=

    Digital exclusion in older adults: A scoping review , author=. International Journal of Nursing Studies , volume=. 2025 , publisher=

    2025

  12. [20]

    5 Alarm Fire in Tai Po , year =

    Government Departments Go All Out to Fight No. 5 Alarm Fire in Tai Po , year =

  13. [21]

    China Daily , year =

    Guo, Yanqi , title =. China Daily , year =

  14. [22]

    Nature Machine Intelligence , volume=

    LLM-based agentic systems in medicine and healthcare , author=. Nature Machine Intelligence , volume=. 2024 , publisher=

    2024

  15. [23]

    NPJ digital medicine , volume=

    Evaluating large language models as agents in the clinic , author=. NPJ digital medicine , volume=. 2024 , publisher=

    2024

  16. [24]

    Nature Machine Intelligence , volume=

    What large language models know and what people think they know , author=. Nature Machine Intelligence , volume=. 2025 , publisher=

    2025

  17. [25]

    Nature Machine Intelligence , pages=

    Language models cannot reliably distinguish belief from knowledge and fact , author=. Nature Machine Intelligence , pages=. 2025 , publisher=

    2025

  18. [26]

    Nature machine intelligence , volume=

    Augmenting large language models with chemistry tools , author=. Nature machine intelligence , volume=. 2024 , publisher=

    2024

  19. [27]

    Nature Communications , volume=

    Evaluating large language model agents for automation of atomic force microscopy , author=. Nature Communications , volume=. 2025 , publisher=

    2025

  20. [28]

    npj Digital Medicine , volume=

    Large language model agents can use tools to perform clinical calculations , author=. npj Digital Medicine , volume=. 2025 , publisher=

    2025

  21. [29]

    Advances in neural information processing systems , volume=

    Attention is all you need , author=. Advances in neural information processing systems , volume=

  22. [30]

    Advances in neural information processing systems , volume=

    Language models are few-shot learners , author=. Advances in neural information processing systems , volume=

  23. [31]

    Advances in neural information processing systems , volume=

    Chain-of-thought prompting elicits reasoning in large language models , author=. Advances in neural information processing systems , volume=

  24. [32]

    2203.11171 , archivePrefix=

    Self-consistency improves chain of thought reasoning in language models , author=. 2203.11171 , archivePrefix=

    arXiv

  25. [33]

    International conference on machine learning , pages=

    Pal: Program-aided language models , author=. International conference on machine learning , pages=. 2023 , organization=

    2023

  26. [34]

    2211.12588 , archivePrefix=

    Program of thoughts prompting: Disentangling computation from reasoning for numerical reasoning tasks , author=. 2211.12588 , archivePrefix=

    arXiv

  27. [35]

    Advances in neural information processing systems , volume=

    Tree of thoughts: Deliberate problem solving with large language models , author=. Advances in neural information processing systems , volume=

  28. [36]

    2112.09332 , archivePrefix=

    Webgpt: Browser-assisted question-answering with human feedback , author=. 2112.09332 , archivePrefix=

    arXiv

  29. [37]

    Advances in Neural Information Processing Systems , volume=

    Hugginggpt: Solving ai tasks with chatgpt and its friends in hugging face , author=. Advances in Neural Information Processing Systems , volume=

  30. [38]

    The twelfth international conference on learning representations , year=

    MetaGPT: Meta programming for a multi-agent collaborative framework , author=. The twelfth international conference on learning representations , year=

  31. [39]

    Advances in neural information processing systems , volume=

    Camel: Communicative agents for" mind" exploration of large language model society , author=. Advances in neural information processing systems , volume=

  32. [40]

    First conference on language modeling , year=

    Autogen: Enabling next-gen LLM applications via multi-agent conversations , author=. First conference on language modeling , year=

  33. [41]

    2305.16291 , archivePrefix=

    Voyager: An open-ended embodied agent with large language models , author=. 2305.16291 , archivePrefix=

    arXiv

  34. [42]

    Qin, Yujia and Liang, Shihao and Ye, Yining and Zhu, Kunlun and Yan, Lan and Lu, Yaxi and Lin, Yankai and Cong, Xin and Tang, Xiangru and Qian, Bill and others , booktitle=

  35. [43]

    2308.03688 , archivePrefix=

    Agentbench: Evaluating llms as agents , author=. 2308.03688 , archivePrefix=

    arXiv

  36. [44]

    Jimenez, Carlos E and Yang, John and Wettig, Alexander and Yao, Shunyu and Pei, Kexin and Press, Ofir and Narasimhan, Karthik , booktitle=

  37. [45]

    Advances in neural information processing systems , volume=

    Reflexion: Language agents with verbal reinforcement learning , author=. Advances in neural information processing systems , volume=

  38. [46]

    Besta, Maciej and Blach, Nils and Kubicek, Ales and Gerstenberger, Robert and Podstawski, Michal and Gianinazzi, Lukas and Gajda, Joanna and Lehmann, Tomasz and Niewiadomski, Hubert and Nyczyk, Piotr and others , booktitle=

  39. [47]

    Findings of the Association for Computational Linguistics: ACL 2025 , pages=

    Data interpreter: An llm agent for data science , author=. Findings of the Association for Computational Linguistics: ACL 2025 , pages=

    2025

  40. [48]

    International Journal of Machine Learning and Cybernetics , volume=

    Large language models for medicine: a survey , author=. International Journal of Machine Learning and Cybernetics , volume=. 2025 , publisher=

    2025

  41. [49]

    Frontiers of Computer Science , volume=

    A survey on large language model based autonomous agents , author=. Frontiers of Computer Science , volume=. 2024 , publisher=

    2024

  42. [50]

    Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    Enhancing Open-Domain Task-Solving Capability of LLMs via Autonomous Tool Integration from GitHub , author=. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

  43. [51]

    2512.07436 , archivePrefix=

    LocalSearchBench: Benchmarking Agentic Search in Real-World Local Life Services , author=. 2512.07436 , archivePrefix=

    arXiv

  44. [52]

    Proceedings of the CHI Conference on Human Factors in Computing Systems , pages=

    Reducing the Search Space on Demand Helps Older Adults Find Mobile UI Features Quickly, on Par with Younger Adults , author=. Proceedings of the CHI Conference on Human Factors in Computing Systems , pages=. 2024 , doi=

    2024

  45. [53]

    Proceedings of the CHI Conference on Human Factors in Computing Systems , pages=

    AppAgent: Multimodal Agents as Smartphone Users , author=. Proceedings of the CHI Conference on Human Factors in Computing Systems , pages=. 2025 , doi=

    2025

  46. [54]

    2401.16158 , archivePrefix=

    Mobile-Agent: Autonomous Multi-Modal Mobile Device Agent with Visual Perception , author=. 2401.16158 , archivePrefix=

    arXiv

  47. [55]

    2403.00813 , archivePrefix=

    UrbanGPT: Spatio-Temporal Large Language Models , author=. 2403.00813 , archivePrefix=

    arXiv

  48. [56]

    2406.13948 , archivePrefix=

    CityGPT: Empowering Urban Spatial Cognition of Large Language Models , author=. 2406.13948 , archivePrefix=

    arXiv

  49. [57]

    2402.06861 , archivePrefix=

    UrbanKGent: A Unified Large Language Model Agent Framework for Urban Knowledge Graph Construction , author=. 2402.06861 , archivePrefix=

    arXiv

  50. [58]

    2312.16044 , archivePrefix=

    LLMLight: Large Language Models as Traffic Signal Control Agents , author=. 2312.16044 , archivePrefix=

    arXiv

  51. [59]

    and Zhang, Hao and Gonzalez, Joseph E

    Zheng, Lianmin and Chiang, Wei-Lin and Sheng, Ying and Zhuang, Siyuan and Wu, Zhanghao and Zhuang, Yonghao and Lin, Zi and Li, Zhuohan and Li, Dacheng and Xing, Eric P. and Zhang, Hao and Gonzalez, Joseph E. and Stoica, Ion , booktitle=. Judging

  52. [60]

    2305.17926 , archivePrefix=

    Large Language Models are not Fair Evaluators , author=. 2305.17926 , archivePrefix=

    arXiv

  53. [61]

    IET Smart Cities , volume=

    City brain: practice of large-scale artificial intelligence in the real world , author=. IET Smart Cities , volume=. 2019 , publisher=

    2019

  54. [62]

    Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (ACL) , pages=

    Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Models , author=. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (ACL) , pages=

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.