{"work":{"id":"8a0a1d85-83c9-47ca-a505-2d91d244472a","openalex_id":null,"doi":null,"arxiv_id":"2603.22435","raw_key":null,"title":"CaP-X: A Framework for Benchmarking and Improving Coding Agents for Robot Manipulation","authors":null,"authors_text":"Max Fu, Justin Yu, Karim El-Refai, Ethan Kou, Haoru Xue, Huang Huang, Wenli Xiao, Guanzhi Wang, Fei-Fei Li, Guanya Shi, et al","year":2026,"venue":"cs.RO","abstract":"\"Code-as-Policy\" considers how executable code can complement data-intensive Vision-Language-Action (VLA) methods, yet their effectiveness as autonomous controllers for embodied manipulation remains underexplored. We present CaP-X, an open-access framework for systematically studying Code-as-Policy agents in robot manipulation. At its core is CaP-Gym, an interactive environment in which agents control robots by synthesizing and executing programs that compose perception and control primitives. Building on this foundation, CaP-Bench evaluates frontier language and vision-language models across varying levels of abstraction, interaction, and perceptual grounding. Across 12 models, CaP-Bench reveals a consistent trend: performance improves with human-crafted abstractions but degrades as these priors are removed, exposing a dependence on designer scaffolding. At the same time, we observe that this gap can be mitigated through scaling agentic test-time computation--through multi-turn interaction, structured execution feedback, visual differencing, automatic skill synthesis, and ensembled reasoning--substantially improves robustness even when agents operate over low-level primitives. These findings allow us to derive CaP-Agent0, a training-free framework that recovers human-level reliability on several manipulation tasks in simulation and on real embodiments. We further introduce CaP-RL, showing reinforcement learning with verifiable rewards improves success rates and transfers from sim2real with minimal gap. Together, CaP-X provides a principled, open-access platform for advancing embodied coding agents.","external_url":"https://arxiv.org/abs/2603.22435","cited_by_count":null,"metadata_source":"pith","metadata_fetched_at":"2026-07-10T07:26:54.469980+00:00","pith_arxiv_id":"2603.22435","created_at":"2026-05-11T08:56:02.864868+00:00","updated_at":"2026-07-10T07:26:54.469980+00:00","title_quality_ok":true,"display_title":"CaP-X: A framework for benchmarking and improving coding agents for robot manipulation","render_title":"CaP-X: A framework for benchmarking and improving coding agents for robot manipulation"},"hub":{"state":{"work_id":"8a0a1d85-83c9-47ca-a505-2d91d244472a","tier":"hub","tier_reason":"10+ Pith inbound or 1,000+ external citations","pith_inbound_count":15,"external_cited_by_count":null,"distinct_field_count":3,"first_pith_cited_at":"2026-04-11T08:33:14+00:00","last_pith_cited_at":"2026-07-09T13:08:54+00:00","author_build_status":"not_needed","summary_status":"needed","contexts_status":"needed","graph_status":"needed","ask_index_status":"not_needed","reader_status":"not_needed","recognition_status":"not_needed","updated_at":"2026-08-21T01:40:11.192121+00:00","tier_text":"hub"},"tier":"hub","role_counts":[{"context_role":"background","n":2}],"polarity_counts":[{"context_polarity":"background","n":2}],"runs":{},"summary":{},"graph":{},"authors":[]}}