Pith. sign in

REVIEW 3 major objections 5 minor 27 references

Business Arena: Benchmarking LLM Agents in a Realistic Marketplace

T0 review · 3 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Business Arena claims to measure business intelligence in a realistic simulated marketplace, and finds the best LLM agents still fall far behind expert strategies.

desk verdict A serious benchmark for end-to-end business agents with solid internal reliability, but the realism claim is undercut by an engineered demand signal and the lack of released artifacts; worth engaging and sending to peer review. read the letter →

arxiv 2608.08621 v1 pith:3TDWXRW7 submitted 2026-08-09 cs.AI

Yijun Pan , Yukun Lian , Kunyu Shi , Junbo Li , Hongwei Xue , Sicong Xie , Guannan Zhang , Xiaoying Xing This is my paper · ORCID
classification cs.AI
keywords LLMagentsbusinessintelligenceagentbenchmarkmarketplacesimulationcross-bordere-commerceactionattributionmechanismablationlong-horizonevaluation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Business Arena is a simulated cross-border B2B marketplace where an AI agent runs a shop for 30 simulated days, buying from suppliers, setting prices, serving buyers, and satisfying compliance obligations before trading. The paper's central claim is that this environment preserves the four challenges that make real business difficult—noisy evidence, delayed and coupled feedback, a changing market, and persistent obligations—and that strong performance in the arena reflects genuine business intelligence rather than simulator-specific shortcuts. The claim matters because existing agent benchmarks measure fixed, verifiable tasks, while running a business is open-ended, capital-at-risk, and has no single correct answer. If the claim is right, Business Arena offers a controlled way to measure and improve end-to-end business agents before they are trusted with real money.

What carries the argument

The load-bearing object is the arena itself: a 30-day, data-grounded marketplace simulator with hidden state, scripted competitor archetypes, a financial ledger, compliance gates, and a save-fork-load stateful evaluation harness. Around this core, the diagnostic stack—an expert-designed strategy reserve that estimates available opportunity, skill-level metrics that expose operating styles, action-level attribution that traces gains and losses to decisions, and mechanism ablations that test intended behavior against neglect and misuse—is what turns a single profit number into evidence about business capability. The four design mechanisms of noisy evidence, delayed consequences, a changing market, and persistent obligations are what make the scores interpretable as business intelligence rather than as routine workflow completion.

What would settle it

Run the same benchmark with the raw, unprocessed Google Trends series substituted for the post-processed signal: if evidence-guided policies no longer beat the blind baseline by tens of thousands of dollars in final net worth, the claim that scores reflect genuine business intelligence rather than an engineered signal would be undercut.

Watch

Extended reading notes

Core claim

Business Arena places one LLM-operated shop inside a simulated cross-border B2B marketplace with 965 supplier offers across 135 SKUs, more than 60 tools, a persistent workspace, and scripted competitor sellers, over a 30-day episode. The central discovery is that the environment separates models sharply: mean final net worth across 15 frontier models ranges from $188,488 to $20,856, a $9.0\times$ gap, and 51% of runs finish below the $80,000 starting capital. The strongest expert-designed strategy reaches $436,195 in the same world, more than twice the best model mean, showing available opportunity the agents are not yet capturing. Mechanism ablations over portfolio selection, market events, pricing, tariffs, and customer service show that evidence-guided behavior beats both neglect and misuse, which the paper takes as evidence that higher arena scores reflect genuine business intelligence rather than simulator-specific shortcuts.

Load-bearing premise

The claim that higher arena scores reflect genuine business intelligence depends on hand-set simulator parameters and on a post-processed Google Trends signal that is deliberately correlated with hidden demand; if that signal were not so clean, evidence-guided agents might not outperform blind ones, and the measured capability gap could shrink.

Editorial extensions

If this is right

  • Frontier LLM agents are not yet reliable enough for autonomous business operation: 51% of runs lose money, and only four of fifteen models preserve starting capital in every trial.
  • The best model mean ($188,488) trails the best expert-designed strategy ($436,195) by more than a factor of two, indicating substantial headroom in end-to-end business operation.
  • Mechanism ablations show that evidence-guided behavior outperforms both neglect and misuse across five tested mechanisms, supporting the interpretation that higher scores reflect genuine business intelligence.
  • Skill-level metrics reveal stable operating styles—premium sellers, volume wholesalers, customer-service specialists—that a single final score hides, making targeted diagnosis and training-data construction possible.
  • Stateful save-fork-load evaluation enables test-time compute scaling: five-day trace search beat daily search and independent runs under the same rollout budget, evidence that the arena's feedback is genuinely delayed.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper, a natural stress test is to replace the post-processed Google Trends signal with raw, unprocessed public data; because Appendix C reveals the signal is engineered to correlate with hidden demand, the measured capability gap may shrink under real-world noise.
  • Beyond the paper, the action-level attribution chains could be used directly as dense credit assignment for reinforcement learning, converting each sourcing, pricing, and recovery action into a reward signal.
  • Beyond the paper, a normalized opportunity-capture score—model final net worth divided by the best expert-strategy final net worth—would make the benchmark comparable across different world seeds and future arena versions.
  • Beyond the paper, the arena currently tests one business setting (cross-border B2B sourcing), so the generality of the business-intelligence claim to retail, services, or manufacturing remains an open question.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. Business Arena is a simulated cross-border B2B marketplace in which an LLM agent operates a shop over 30 simulated days, with supplier data grounded in Alibaba.com and demand, tariffs, and events calibrated from public sources. The paper evaluates 15 frontier models across 10 runs each, reporting a 9.0x spread in mean final net worth, with 51% of runs losing money, and compares model performance to a library of expert-designed strategies. It contributes diagnostic skill-level metrics, action-level attribution, stateful checkpointing, and nine mechanism ablations. The central claims are that the arena preserves the challenges of real business and that higher arena scores reflect genuine business intelligence rather than shortcuts, leaving large headroom between frontier LLM agents and expert strategies.

Significance. If the realism claim were fully supported, this would be a valuable benchmark for long-horizon, open-ended agent evaluation: it combines noisy evidence, delayed feedback, non-stationarity, and persistent obligations in one testbed, and the evaluation methodology is careful. The reliability analysis (ICC = 0.944 for ten-run means; split-half rho = 0.898) and the nine mechanism ablations with negative controls are strengths, as are the stateful save-fork-load pipeline and action-level attribution. However, the external validity of the demand signal and the reference-strategy headroom estimate are not yet established; the within-benchmark evidence is strong, but the realistic and trustworthy claim remains conditional.

major comments (3)
  1. [Appendix C and Appendix G, Table 1] The demand signal is post-processed to correlate with oracle demand: Appendix C states 'We conduct minor post-processing of google trends data so that the results is correlated with oracle demand,' and Appendix G reports a pooled correlation of 0.972 with 93.3% country-category identification. This engineered correlation means the evidence-guided sourcing advantage of +$63.6k in Table 1 is measured under a public signal nearly as informative as the hidden demand state. The manuscript does not report the raw Google Trends correlation before post-processing, nor does it provide a noise model, temporal lag structure, or category-specific degradation. Because real public demand signals are substantially noisier and time-lagged, the measured capability gap and the claimed headroom could shrink or reorder if the signal were cleaned to a realistic noise level. The authors should add a raw-correlation baseline, a sensitivity analysis across noise levels, or validation against out-of-sample real demand series before claiming that the arena preserves the noisy-evidence challenge. This issue is load-bearing for the Section 6.5 conclusion that higher scores reflect genuine business intelligence.
  2. [Section 6.5 and Appendix J] The expert-designed strategies, including the Bayesian export compounder reaching $436,195, are authored by the same team with full knowledge of the arena's mechanisms and NPC behavior. Although they operate only on agent-visible information, this reference standard may encode privileged knowledge of which mechanisms matter, making the substantial-headroom conclusion partly internal to the authors' design. The paper should test the robustness of the headroom estimate, for example by using independently authored strategies or by varying NPC population and world seeds and showing that the expert advantage is stable. Without such evidence, the conclusion that frontier models fall substantially behind human-designed strategies is not independently validated.
  3. [Section 6.5 and Appendix G] The mechanism ablations compare hand-coded intended policies against hand-coded neglect and misuse variants; they demonstrate that the arena rewards the intended mechanisms, but they do not directly establish that the observed LLM leaderboard differences are caused by those mechanisms. The paper states that 'these ablations support interpreting higher Business Arena scores as evidence of stronger business intelligence,' yet the policies that earn higher scores in the ablations are not the LLM agents. To bridge this gap, the authors should explicitly acknowledge this limitation and, where possible, correlate model-level scores with the corresponding skill-level metrics (e.g., demand-inference accuracy or pricing precision) to show that the same mechanisms drive both the ablation ladder and the model ranking.
minor comments (5)
  1. [Appendix C] The sentence 'We conduct minor post-processing of google trends data so that the results is correlated with oracle demand' contains a grammatical error; 'results is' should be 'results are'.
  2. [Section 4 and Appendix J] The word 'capitol' appears in 'capitol commitments' (Section 4) and 'redeploy capitol' (Appendix J); both should be 'capital'.
  3. [Appendix D] The text mentions '10 baselines (one per archetype, used for alpha computation)' but never defines what alpha is; please specify this quantity or remove the reference.
  4. [Appendix M] The sentence 'The contrast shows the what we expect from models' is ungrammatical and should be rephrased, for example to 'The contrast shows what we expect from models.'
  5. [References] Reference [4] is listed as 'arXiv preprint, 2026' without author names; please complete the citation so readers can locate the work.

Circularity Check

1 steps flagged · score 4.0 of 10

Demand-signal ablation is self-definitional: Google Trends is post-processed to correlate with oracle demand (pooled ρ=0.972), so the evidence-guided advantage is guaranteed by construction rather than an empirical property of real public signals.

  1. fitted input called prediction [Appendix C ('Realistically-grounded Demand'); relied on in Section 6.5 Table 1 and Appendix G demand-inference ablation]
    "We conduct minor post-processing of google trends data so that the results is correlated with oracle demand, thus testing the agent's ability to do numerical analysis to estimate demand trends."

    By construction, the agent-visible Google Trends signal is post-processed to correlate with the hidden oracle demand it is supposed to infer. The demand-inference ablation then reports pooled ρ=0.972 for 'public evidence' and a $63.6k advantage for evidence-guided sourcing over blind bulk buying (Sec. 6.5, App. G). That correlation is the design target of the post-processing, not an empirical property of real public data. No raw-correlation baseline, noise model, or lag structure is reported, so the ablation only verifies the arena's internal consistency.

full rationale

The derivation is mostly self-contained: 15 frontier models run in an isolated sandbox with a protected information boundary; the leaderboard means, ICC reliability, split-half rank correlation, and action-level attribution are computed from recorded interactions, not from fitted parameters. The expert-designed strategies are internal baselines, not predictions, and no load-bearing self-citation chain appears. The one genuinely circular element is the demand-signal construction. Appendix C states that Google Trends data are post-processed so that the results correlate with oracle demand; Appendix G then reports that the public evidence recovers the hidden demand ranking at ρ=0.972 and that evidence-guided sourcing beats blind policies by $63.6k. Because the signal was engineered to be nearly as informative as the oracle, the demand-inference ablation is a tautology with respect to the realism claim: it demonstrates that a signal designed to reveal hidden demand helps agents, but says nothing about whether real-world public signals are that informative. The paper does not provide a raw-signal baseline or noise model that would support transfer to noisier real markets. This affects the 'realistic and trustworthy testbed' conclusion (Section 6.5) but not the internal model ordering or the skill-level/attribution analyses, so the overall circularity is partial rather than total. Score 4 reflects one load-bearing manufactured input in an otherwise independent evaluation.

Assumptions & free parameters 6 free parameters · 4 assumptions · 2 invented entities

The benchmark's conclusions rest on many design choices: simulator parameters, post-processed signals, authored baseline strategies, and validity assumptions. These are not fitted to model outputs, so the core comparison is not circular, but they are unverified from outside.

free parameters (6)
  • Daily fixed overhead = $200/day
    Appendix F Table 4 sets $200 daily overhead for the paper condition; directly affects capital erosion and the cost of idle cash.
  • Final settlement discounts = 0.97 escrow, 0.97 receivables, 0.85 inventory
    Appendix F defines FinalAssets with these factors; changes salvage value and terminal net worth.
  • Compliance fine schedule = max($500, 0.15 × order value) × min(k,5)
    Appendix E; chosen by authors to make compliance economically consequential.
  • Google Trends post-processing correlation = correlated with oracle demand (ρ=0.972)
    Appendix C; public signal is deliberately engineered to align with hidden demand.
  • NPC population distribution = micro 45%, small 30%, mid 18%, large 5%, enterprise 2%
    Appendix D; chosen to mirror a skewed marketplace and affects competitive pressure.
  • Festival demand caps = capped category-specific changes
    Appendix C; converted evidence into capped demand multipliers so festivals matter but do not decide outcomes.
assumptions (4)
  • domain assumption The arena's mechanisms are a valid operationalization of real business challenges
    Section 3.2 claims the mechanisms preserve the key abstractions; no external validation against live business outcomes.
  • ad hoc to paper Post-processed Google Trends signal is a legitimate public demand signal
    Appendix C states post-processing is applied specifically to make the signal correlate with oracle demand.
  • ad hoc to paper Expert-designed strategies represent the available opportunity in the arena
    Section 4.1 and Appendix J: strategies are authored by the same team with full knowledge of the arena.
  • standard math Standard reliability statistics (ICC, split-half) are applicable to these score distributions
    Appendix H assumes approximately normal variance components for ICC; ten runs per model is a small sample.
invented entities (2)
  • Hidden oracle demand state
    purpose: Ground truth for demand-inference ablations
    Introduced by the authors as an upper bound; not observable by agents or from outside data.
  • NPC seller archetypes
    purpose: Simulate competing sellers with ten pricing strategies
    Defined in Appendix D; chosen by authors, not validated against real seller distributions.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Business Arena: Benchmarking LLM Agents in a Realistic Marketplace." pith.science (2026). https://pith.science/paper/3TDWXRW7

@misc{pith2026260808621,
  author       = {Pith},
  title        = {Pith review of: Business Arena: Benchmarking LLM Agents in a Realistic Marketplace},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3TDWXRW7}},
  note         = {Machine review of arXiv:2608.08621}
}
read the original abstract

Running a business is a challenging form of intelligent work. Operators must infer opportunities from partial signals, commit capital under uncertainty, adapt to delayed outcomes in a changing market, and satisfy regulatory obligations before trading legally. Frontier LLM agents can increasingly complete complex workflows, yet business-related capabilities are rarely evaluated in existing agent benchmarks. We introduce \textbf{Business Arena}, a controlled environment where an AI agent runs a cross-border shop, buying from suppliers and selling to buyers over a long horizon. We ground the arena in real Alibaba.com sourcing data and market conditions calibrated from authoritative sources. Delayed and coupled consequences make individual business decisions difficult to judge, but their combined outcome is measurable through profit. Because profit alone cannot explain why an agent succeeds or fails, we compare agents with human-designed strategies to estimate available opportunity, use skill-level metrics to reveal underlying strengths and weaknesses, and trace realized gains and losses to the actions that produced them. We use mechanism ablations to establish that strong results reflect genuine business intelligence rather than neglect or simulator-specific shortcuts. We evaluate 15 frontier models and find a ninefold difference in mean final net worth. Even the best model falls behind human-designed strategies, indicating that business operation remains challenging for LLM agents. Skill-level analysis reveals operating styles, from margin-focused premium sellers to high-turnover wholesalers and customer-service specialists, while action-level attribution identifies the sourcing, pricing, and recovery decisions that create or destroy value. Together, Business Arena takes a first step toward a realistic and trustworthy testbed for evaluating end-to-end business agents.

Figures

Figures reproduced from arXiv: 2608.08621 by the authors.

Figure 1
Figure 1. Model performance in Business Arena. Over the same long horizon, the strongest models more than double their capital, a middle group earns modest returns, and the weakest ones finish with less than they started. What prevents such agents from operating real businesses today? Unlike other workflows, business requires high-stakes decisions under four major challenges. Evidence is noisy: true demand, competitor intenti… view at source ↗
Figure 2
Figure 2. Overview of arena design. The agent selects markets, purchases inventory, prices and lists products, learns from sales, and adapts its operation. It acts on partial observations while supplier disruptions, competitor repricing, and demand shifts create a changing market. Meanwhile, operational obligations persist and economic feedback remains delayed. Incomplete and noisy evidence. Real markets rarely reveal a groun… view at source ↗
Figure 3
Figure 3. Main leaderboard over 15 model families, averaged across ten runs under the same world condition. Dashed lines denote expert-designed strategies that use only agent-visible information. Substantial headroom remains in Business Arena. The strongest expert-designed strategy reaches $436,195 in the same world, more than twice the best model mean. It achieves this by coordinating market evidence with portfolio selection… view at source ↗
Figures from the paper (16 more)
Figure 4
Figure 4. Figure 4: Model diagnostic profiles. Models exhibit different strengths across operating fluency, capital deployment, selling, customer interaction, and compliance. Colors indicate cohort-relative performance from weaker to stronger. (a) Capital deployment and inventory turnover…
Figure 5
Figure 5. Figure 5: Core operating trade-offs. Successful agents identify opportunities and deploy capital into inventory that will sell (left), then preserve margin without pricing themselves out of the market (right). Crosses denote model-family means and lighter points individual runs.…
Figure 6
Figure 6. Figure 6: Compliance exposes deployment-critical reliability failures (left), while customer service reveals specialized strengths that do not follow the aggregate leaderboard (right). while DeepSeek V4 Pro averages 17.4 violations and $39,650. These failures show that profitabi…
Figure 7
Figure 7. Figure 7: GPT-5.6 Sol behaves like a Volume Wholesaler. Its self-defined sourcing standards reject risky products, prioritize high-margin routes, and diversify inventory while retaining cash, supporting aggressive capital deployment and high sell-through. The preceding metrics c…
Figure 8
Figure 8. Figure 8: Gemini 3.1 Pro resembles a Premium House. Its route-aware repricer incorporates supplier cost, freight, tariffs, and competition while enforcing a 15% margin floor, producing higher margins at the cost of lower sell-through. We have shown that Gemini 3.1 Pro achieves t…
Figure 9
Figure 9. Figure 9: GPT-5.5 demonstrates adaptive recovery. After detecting zero cash and slow-moving inventory, it liquidates stock, reduces advertising, adopts FOB terms, and resets prices to release trapped capital and continue operating. applies one price across markets, omits shippin…
Figure 10
Figure 10. Figure 10: Realized value attribution within one trajectory. Two decisions by Gemini 3.5 Flash produce opposite outcomes: route-specific landed-cost reasoning preserves margin for SH-04, while underestimated delivery cost makes a TB-03 order loss-making. Mechanism Intended behav…
Figure 11
Figure 11. Figure 11: Repeated-run leaderboard performance. Each point represents one finalized episode; diamonds denote model means, and horizontal lines show two-sided 95% confidence intervals over ten runs. The dashed line marks the $80,000 starting capital. Persistent differences betwe…
Figure 12
Figure 12. Figure 12: gives an example of the seller-buyer interface in Business Arena (note that this is a dedicated illustration of the marketplace, business arena involve no evaluation of GUI agent use) [PITH_FULL_IMAGE:figures/full_fig_p026_12.png]
Figure 13
Figure 13. Figure 13: Expert-designed strategy decision architecture. Agent-visible evidence and prior operating history are converted into beliefs of the market, which inform a coordinating capital and portfolio plan. This plan constrains the downstream operating tree, whose actions are e…
Figure 14
Figure 14. Figure 14: Comparison of expert-designed strategies and selected models. The strongest expert strategies recycle returned cash across more SKUs while preserving higher margins and avoiding compliance fines. Inventory-light broker. This strategy minimizes ownership risk. It opens…
Figure 15
Figure 15. Figure 15: Checkpoint cadence matters when scaling long-horizon business operation. The left panel shows the continuations considered by five-day trace search, its retained lineages, the trajectory retained by daily search, and six independent full runs. Intermediate values thro…
Figure 16
Figure 16. Figure 16: Different decisions under the same market shock. Gemini 3.5 Flash incorporates the tariff change into route-specific pricing and market access, while MiniMax M2.5 reopens the affected route without refreshing its cost assumptions. Each decision is linked to its realiz…
Figure 17
Figure 17. Figure 17: Compliance behavior of GPT-5.6 Sol and DeepSeek V4 Pro. GPT-5.6 Sol checks certification requirements, applies for the required credential, and verifies approval before reopening trade. DeepSeek V4 Pro recognizes repeated compliance violations but does not take correc…
Figure 18
Figure 18. Figure 18: Customer-service behavior of GLM-5.2 and Gemini 3.1 Pro. GLM-5.2 addresses the buyer’s specific commercial questions with concrete, factually-supported information, while Gemini 3.1 Pro gives a generic response that omits key decision criteria and loses the inquiry. 3…
Figure 19
Figure 19. Figure 19: Advertising strategies of GPT-5.6 Sol and GLM-5.2. Both models observe contribution￾level feedback, but GPT-5.6 Sol adapts its advertising allocation by pruning unprofitable campaigns and refining its strategies. GLM-5.2 recognizes negative returns but does not consis…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

27 extracted references · 20 canonical work pages

  1. [1]

    DMTCP: transparent checkpointing for cluster computa- tions and the desktop

    Jason Ansel, Kapil Arya, and Gene Cooperman. DMTCP: transparent checkpointing for cluster computa- tions and the desktop. In2009 IEEE International Symposium on Parallel and Distributed Processing, 1–12. 2009. doi:10.1109/IPDPS.2009.5161063

    arXiv 2009

  2. [2]

    Vending-bench: a benchmark for long-term coherence of autonomous agents.arXiv preprint arXiv:2502.15840, 2025

    Axel Backlund and Lukas Petersson. Vending-bench: a benchmark for long-term coherence of autonomous agents.arXiv preprint arXiv:2502.15840, 2025

    arXiv 2025

  3. [3]

    CEO-Bench: can agents play the long game?arXiv preprint arXiv:2606.18543, 2026

    Haozhe Chen, Karthik Narasimhan, and Zhuang Liu. CEO-Bench: can agents play the long game?arXiv preprint arXiv:2606.18543, 2026

    arXiv 2026

  4. [4]

    SWE-Marathon: long-horizon software engineering tasks for LLM agents.arXiv preprint, 2026

    Aditya Desai and others. SWE-Marathon: long-horizon software engineering tasks for LLM agents.arXiv preprint, 2026

    work page 2026

  5. [5]

    DeltaBox: scaling stateful AI agents with millisecond-level sandbox checkpoint/rollback.arXiv preprint arXiv:2605.22781, 2026

    Yunpeng Dong, Jingkai He, Yuze Hou, Dong Du, Zhonghu Xu, Si Yu, Yubin Xia, and Haibo Chen. DeltaBox: scaling stateful AI agents with millisecond-level sandbox checkpoint/rollback.arXiv preprint arXiv:2605.22781, 2026

    arXiv 2026

  6. [6]

    YC-Bench: benchmarking AI agents for long-term planning and consistent execution.arXiv preprint arXiv:2604.01212, 2026

    Muyu He, Adit Jain, Anand Kumar, Vincent Tu, Soumyadeep Bakshi, Sachin Patro, and Nazneen Rajani. YC-Bench: benchmarking AI agents for long-term planning and consistent execution.arXiv preprint arXiv:2604.01212, 2026

    arXiv 2026

  7. [7]

    SWE-bench: can language models resolve real-world GitHub issues? InICLR

    Carlos E Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik Narasimhan. SWE-bench: can language models resolve real-world GitHub issues? InICLR. 2024

    work page 2024

  8. [8]

    The role of management games and simulations in education and research

    Bernard Keys and Joseph Wolfe. The role of management games and simulations in education and research. Journal of Management, 16(2):307–336, 1990

    work page 1990

Show all 27 references
  1. [9]

    Shrout and Joseph L

    Patrick E. Shrout and Joseph L. Fleiss. Intraclass correlations: uses in assessing rater reliability.Psycho- logical Bulletin, 86(2):420–428, 1979. doi:10.1037/0033-2909.86.2.420

    1979 doi

  2. [10]

    Herbert A. Simon. A behavioral model of rational choice.The Quarterly Journal of Economics, 69(1):99– 118, 1955

    1955

  3. [11]

    John D. Sterman. Modeling managerial behavior: misperceptions of feedback in a dynamic decision making experiment.Management Science, 35(3):321–339, 1989

    1989

  4. [12]

    Teece, Gary Pisano, and Amy Shuen

    David J. Teece, Gary Pisano, and Amy Shuen. Dynamic capabilities and strategic management.Strategic Management Journal, 18(7):509–533, 1997

    1997

  5. [13]

    Agent-based computational economics: a constructive approach to economic theory

    Leigh Tesfatsion. Agent-based computational economics: a constructive approach to economic theory. In Handbook of Computational Economics, volume 2, pages 831–880. Elsevier, 2006

    2006

  6. [14]

    ShoppingBench: a real-world intent-grounded shopping benchmark for LLM-based agents.AAAI, 2024

    Yinuo Wang, Chuanfeng Xiao, and others. ShoppingBench: a real-world intent-grounded shopping benchmark for LLM-based agents.AAAI, 2024

    2024

  7. [15]

    Crab: a semantics-aware check- point/restore runtime for agent sandboxes.arXiv preprint arXiv:2604.28138, 2026

    Tianyuan Wu, Chaokun Chang, Lunxi Cao, Wei Gao, and Wei Wang. Crab: a semantics-aware check- point/restore runtime for agent sandboxes.arXiv preprint arXiv:2604.28138, 2026

    2026 arXiv

  8. [16]

    τ-bench: a benchmark for tool-agent- user interaction in real-world domains

    Shunyu Yao, Noah Shinn, Pedram Razavi, and Karthik Narasimhan. τ-bench: a benchmark for tool-agent- user interaction in real-world domains. InICLR. 2025

    2025

  9. [17]

    WebArena: a realistic web environment for building autonomous agents

    Shuyan Zhou, Frank F Xu, Hao Zhu, Xuhui Zhou, Robert Lo, Abishek Sridhar, Xianyi Cheng, Tianyue Ou, Yonatan Bisk, Daniel Fried, Uri Alon, and Graham Neubig. WebArena: a realistic web environment for building autonomous agents. InICLR. 2024

    2024

  10. [18]

    The 2024 holiday season set new records for ecommerce

    Adobe Digital Insights. The 2024 holiday season set new records for ecommerce. 2025. URL: https: //business.adobe.com/uk/blog/2024-holiday-season-set-new-records-for-ecommerce

    2024

  11. [19]

    Holiday commerce data and shopping trends

    Criteo. Holiday commerce data and shopping trends. URL:https://www.criteo.com/insights/

  12. [20]

    Turnover and volume of sales in wholesale and retail trade: monthly data

    Eurostat. Turnover and volume of sales in wholesale and retail trade: monthly data. Dataset sts_trtu_m. URL:https://ec.europa.eu/eurostat/databrowser/view/sts_trtu_m/default/table

  13. [21]

    Federal Reserve Bank of St. Louis. Fred api: economic data observations. URL:https://fred.stlou isfed.org/docs/api/fred/

  14. [22]

    Google trends

    Google. Google trends. URL:https://trends.google.com/. 15

  15. [23]

    European holiday spending outlook

    Mastercard Economics Institute. European holiday spending outlook. 2024. URL: https://www.mast ercard.com/news/europe/en/newsroom/press-releases/en/2024/european-shoppers-set -to-celebrate-with-spending-on-fashion-gadgets-and-travel-this-festive-season/

    2024

  16. [24]

    Total retail sales of consumer goods

    National Bureau of Statistics of China. Total retail sales of consumer goods. URL: https://www.stats. gov.cn/english/PressRelease/

  17. [25]

    Holiday data and consumer trends

    National Retail Federation. Holiday data and consumer trends. URL: https://nrf.com/research-i nsights/holiday-data-and-trends

  18. [26]

    Census Bureau

    U.S. Census Bureau. Monthly retail trade survey. URL: https://www.census.gov/retail/mrts/a bout_the_surveys.html

  19. [27]

    World integrated trade solution: bilateral tariff technical note

    World Bank. World integrated trade solution: bilateral tariff technical note. WITS Trade and Tariff Database. Tariff data sourced from UNCTAD TRAINS and WTO IDB. URL: https://wits.worldba nk.org/Bilateral-Tariff-Technical-Note.html. 16 A Agent Runtime and Interface Autonomous ...

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.