{"arxiv_id":"2609.38974","title":"RealWorldShop: Benchmarking and Improving Conversational Shopping Agents in Real-World E-commerce","abstract":"Large language models are reshaping ecommerce from static recommenders into interactive shopping assistants, yet real-world shopping requires session-level decision support: users reveal and revise constraints, coordinate multiple goals, and expect product-grounded recommendations over a full conversation. Existing benchmarks are mostly outcome-oriented or execution-oriented, leaving this evolving decision process under-evaluated. We introduce REALWORLDSHOP, a benchmark built on 3.28M grounded products, structured shopping episodes, a profile-grounded and actioncontrolled user simulator, and role-play evaluation. Our analysis shows that current systems produce locally plausible responses but struggle with state tracking, constraint updating, and grounded convergence, especially under ambiguous intent, bundle, and multi-intent scenarios. We further propose REALSHOP_AGENT, an executable session-control framework with explicit state management, shopping-flow control, catalog-grounded retrieval, and runtime guards. Experiments show that REALSHOP_AGENT consistently outperforms strong baselines on REALWORLDSHOP.","primary_cat":"cs.AI","cross_cats":[],"submitted_at":"2026-09-30T04:49:10+00:00","latest_version":1,"source":"arxiv","source_label":"arXiv","source_url":null,"has_pdf":true,"has_source":false,"verdict":"CONDITIONAL","confidence":"MODERATE","one_line_summary":"A session-level benchmark on 3.28M real products shows shopping agents fail at long-horizon state tracking, and an explicit session-control agent raises success from 0.71 to 0.82.","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","upvotes":0,"downvotes":0,"pith_score":0.0,"inbound_count":1,"_links":{"html":"https://pith.science/paper/2609.38974","verdict":"/api/v1/verdicts/2609.38974"}}