Pith. sign in

REVIEW 12 cited by

AgentStudio: A Toolkit for Building General Virtual Agents

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2403.17918 v3 pith:NJYPAW3B submitted 2024-03-26 cs.AI

Longtao Zheng , Zhiyuan Huang , Zhenghai Xue , Xinrun Wang , Bo An , Shuicheng Yan This is my paper
classification cs.AI
keywords toolsagentagentsagentstudiodatasetsenvironmentsgeneralvirtual
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

General virtual agents need to handle multimodal observations, master complex action spaces, and self-improve in dynamic, open-domain environments. However, existing environments are often domain-specific and require complex setups, which limits agent development and evaluation in real-world settings. As a result, current evaluations lack in-depth analyses that decompose fundamental agent capabilities. We introduce AgentStudio, a trinity of environments, tools, and benchmarks to address these issues. AgentStudio provides a lightweight, interactive environment with highly generic observation and action spaces, e.g., video observations and GUI/API actions. It integrates tools for creating online benchmark tasks, annotating GUI elements, and labeling actions in videos. Based on our environment and tools, we curate an online task suite that benchmarks both GUI interactions and function calling with efficient auto-evaluation. We also reorganize existing datasets and collect new ones using our tools to establish three datasets: GroundUI, IDMBench, and CriticBench. These datasets evaluate fundamental agent abilities, including GUI grounding, learning from videos, and success detection, pointing to the desiderata for robust, general, and open-ended virtual agents.

Discussion (0). Sign in to comment.

Forward citations

Cited by 12 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. R-VLM: Region-Aware Vision Language Model for Precise GUI Grounding

    cs.CV 2025-07 conditional novelty 7.0 of 10

    R-VLM improves GUI grounding by combining two-stage zoom-in proposals with an IoU-weighted training loss, raising accuracy by up to 13 absolute points over SeeClick.

  2. AndroidWorld: A Dynamic Benchmarking Environment for Autonomous Agents

    cs.AI 2024-05 accept novelty 7.0 of 10

    AndroidWorld is a dynamic, reproducible Android benchmark that generates unlimited natural-language tasks for autonomous agents and shows current agents succeed on only 30.6 percent of them.

  3. Multi-Head Latent Control: A Unified Interface for LLM Agent Decision Making

    cs.CL 2026-07 conditional novelty 6.0 of 10

    Hidden-state traces of frozen LLMs/VLMs can be read by lightweight trained heads to predict when to defer, clarify, call tools, or abstain, cutting routed inference cost 27–90%.

  4. Learning to Adapt: Self-Improving Web Agent via Cognitive-Aware Exploration

    cs.AI 2026-05 unverdicted novelty 6.0 of 10

    SCALE introduces three adversarial roles (Selector, Predictor, Judger) and a graph exploration method (SCALE-Hop) to enable MLLM-based web agents to self-discover limitations and improve, backed by the SCALE-20k datas...

  5. ClawEnvKit: Automatic Environment Generation for Claw-Like Agents

    cs.AI 2026-04 unverdicted novelty 6.0 of 10

    ClawEnvKit automates generation of diverse verified environments for claw-like agents from natural language, producing the Auto-ClawEval benchmark of 1,040 environments that matches human-curated quality at 13,800x lo...

  6. OS-ATLAS: A Foundation Action Model for Generalist GUI Agents

    cs.CL 2024-10 unverdicted novelty 6.0 of 10

    OS-Atlas, trained on the largest open-source cross-platform GUI grounding corpus of 13 million elements, outperforms prior open-source models on six benchmarks across mobile, desktop, and web platforms.

  7. Agent Skills Should Go Beyond Text: The Case for Visual Skills

    cs.CV 2026-05 unverdicted novelty 5.0 of 10

    The paper proposes that reusable agent skills should incorporate visual elements alongside text, introduces three forms of visual skills and an automatic conversion system, and reports better performance on GUI and vi...

  8. A Comprehensive Survey of Agents for Computer Use: Foundations, Challenges, and Future Directions

    cs.AI 2025-01 unverdicted novelty 5.0 of 10

    A survey of 87 agents for computer use and 33 datasets that introduces a three-dimensional taxonomy across domain, interaction, and agent perspectives and identifies six research gaps.

  9. ClawEnvKit: Automatic Environment Generation for Claw-Like Agents

    cs.AI 2026-04 conditional novelty 4.0 of 10

    EVT improves the RMT backbone by using Euclidean-distance attention decay and 1D token grouping, achieving 86.6% top-1 on ImageNet-1K at 384×384 resolution.

  10. Exploring the Link Between Bayesian Inference and Embodied Intelligence: Toward Open Physical-World Embodied AI Systems

    cs.AI 2025-07 conditional novelty 4.0 of 10

    A position paper arguing that Bayesian inference could become a key design principle for embodied AI in open physical worlds, using Sutton's search-and-learning lens to explain its current absence.

  11. Large Language Model-Brained GUI Agents: A Survey

    cs.AI 2024-11 unverdicted novelty 4.0 of 10

    A survey consolidating frameworks, data practices, large action models, benchmarks, applications, and research gaps in LLM-brained GUI agents.

  12. Personal LLM Agents: Insights and Survey about the Capability, Efficiency and Security

    cs.HC 2024-01 unverdicted novelty 3.0 of 10

    This survey discusses key components and challenges for Personal LLM Agents and reviews solutions for their capability, efficiency, and security.

Pith tools