Recognition: unknown
[COMP25] The Automated Negotiating Agents Competition (ANAC) 2025 Challenges and Results
Pith reviewed 2026-05-10 12:09 UTC · model grok-4.3
The pith
The ANAC 2025 competition identifies challenges in multi-deal and concurrent supply chain negotiations and presents agent results.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The central claim is that the ANAC 2025 competition successfully highlighted key difficulties in multi-deal negotiations and concurrent negotiation for complex supply chains, with the analysis of submitted agents providing insights that inform the next steps in the field.
What carries the argument
The competition framework consisting of multi-deal negotiation scenarios and concurrent negotiation in supply chain management environments, which serves to evaluate and compare different automated agent strategies.
Load-bearing premise
Competition results from artificial negotiation scenarios can be generalized to guide the creation of agents for actual business negotiations.
What would settle it
A real-world test where agents from the competition are deployed in live supply chain negotiations and fail to match the expected performance levels.
read the original abstract
This paper presents the primary research challenges and key findings from the 15th International Automated Negotiating Agents Competition (ANAC 2025), one of the official competitions of IJCAI 2025. We focus on two critical domains: multi-deal negotiations and the development of agents capable of concurrent negotiation within complex supply chain management environments. Furthermore, this work analyzes the results of the competition and outlines strategic directions for future iterations.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper presents the primary research challenges and key findings from the 15th International Automated Negotiating Agents Competition (ANAC 2025), one of the official competitions of IJCAI 2025. It focuses on two domains—multi-deal negotiations and concurrent negotiation within complex supply chain management environments—describes the competition setups and participating agents, analyzes the results, and outlines strategic directions for future iterations.
Significance. This manuscript is significant as a benchmark report that documents empirical agent performance in multi-deal and concurrent negotiation scenarios, which are relevant to practical multi-agent systems applications such as supply chain coordination. Competition reports of this type provide standardized comparisons that track progress in automated negotiation and identify open challenges, fostering community-wide advancement. The emphasis on complex environments adds value, provided the analysis avoids post-hoc biases and the strategic directions are tied directly to observed outcomes.
major comments (2)
- [Results section] Results section: The manuscript states that results were analyzed but does not explicitly describe the pre-registered evaluation metrics, whether any adjustments were made after observing outcomes, or the statistical methods used. This leaves open the possibility of post-hoc selection or unstated assumptions in the competition design, which is load-bearing for the reliability of the key findings and agent rankings.
- [Strategic directions section] Strategic directions section: The outlined directions for future iterations are presented as informed by the competition outcomes, yet the text provides no concrete mapping of competition utilities, information structures, or observed agent behaviors to documented real-world supply chain parameters (e.g., variable lead times or multi-party legal constraints). This weakens the transferability claim for the identified challenges.
minor comments (3)
- [Abstract and Introduction] The abstract and introduction would benefit from a brief table summarizing the two domains, their key parameters, and the number of participating agents to improve readability.
- [Figures] Figure captions for performance plots should explicitly state the axes, the number of runs per agent, and any error bars or statistical significance indicators.
- [Related Work] A short related-work subsection comparing ANAC 2025 domains to prior ANAC editions (e.g., 2023–2024) would better contextualize the novelty of the multi-deal and concurrent supply-chain tracks.
Simulated Author's Rebuttal
We thank the referee for the constructive comments on our manuscript reporting the ANAC 2025 competition results. We have carefully considered the points raised regarding evaluation transparency and real-world transferability, and we address each below with corresponding revisions to strengthen the paper.
read point-by-point responses
-
Referee: [Results section] Results section: The manuscript states that results were analyzed but does not explicitly describe the pre-registered evaluation metrics, whether any adjustments were made after observing outcomes, or the statistical methods used. This leaves open the possibility of post-hoc selection or unstated assumptions in the competition design, which is load-bearing for the reliability of the key findings and agent rankings.
Authors: We appreciate this observation on methodological transparency. The metrics in ANAC 2025 follow the pre-defined competition protocol used in prior editions, consisting of normalized utility scores, agreement rates, and efficiency measures (e.g., rounds to agreement). No adjustments to metrics, rankings, or analysis procedures were performed after observing outcomes; all evaluations adhered to the announced rules. We acknowledge that explicit documentation of this protocol and the statistical approach (means, standard deviations across repeated trials, and non-parametric tests for significance) was omitted from the manuscript. We have added a dedicated subsection to the Results section that states the pre-registered metrics, confirms the absence of post-hoc changes, and details the statistical methods employed. This revision directly addresses the concern and improves reproducibility. revision: yes
-
Referee: [Strategic directions section] Strategic directions section: The outlined directions for future iterations are presented as informed by the competition outcomes, yet the text provides no concrete mapping of competition utilities, information structures, or observed agent behaviors to documented real-world supply chain parameters (e.g., variable lead times or multi-party legal constraints). This weakens the transferability claim for the identified challenges.
Authors: We agree that explicit linkages to real-world supply chain characteristics would strengthen the discussion of future directions. The identified challenges (e.g., handling concurrent deals under incomplete information and balancing multi-party utilities) emerged directly from agent performance in the competition's supply-chain scenarios. We have revised the Strategic directions section to include concrete mappings: for instance, competition utility functions that trade off cost versus delivery time are now explicitly related to variable lead times, while information asymmetry structures are connected to multi-party legal constraints and coordination delays typical in supply chains. These additions preserve the competition's abstracted nature while clarifying transferability without overstating equivalence. revision: yes
Circularity Check
No circularity: descriptive competition report with no derivations or self-referential claims.
full rationale
The paper reports outcomes from the external ANAC 2025 competition, including domain descriptions, agent results, and future directions. No equations, fitted parameters, predictions, or load-bearing self-citations appear in the provided text. All content is observational reporting of independent competition data, with no reduction of claims to inputs by construction. This matches the default expectation of no significant circularity for such papers.
Axiom & Free-Parameter Ledger
Reference graph
Works this paper leans on
-
[1]
[Mohammadet al., 2021 ] Yasser Mohammad, Shinji Nakadai, and Amy Greenwald
Springer International Publishing. [Mohammadet al., 2021 ] Yasser Mohammad, Shinji Nakadai, and Amy Greenwald. Negmas: a platform for situated negotiations. InRecent Advances in Agent-based Negotiation: F ormal Models and Human Aspects, pages 57–75. Springer, 2021. [Mohammadet al., 2025 ] Yasser Mohammad, Shinji Nakadai, and Amy Greenwald. Automated negot...
2021
-
[2]
Springer Nature Switzerland. [Terryet al., 2021 ] J Terry, Benjamin Black, Nathaniel Grammel, Mario Jayakumar, Ananth Hari, Ryan Sullivan, Luis S Santos, Clemens Dieffendahl, Caroline Horsch, Ro- drigo Perez-Vicente, et al. Pettingzoo: Gym for multi- agent reinforcement learning.Advances in Neural Infor- mation Processing Systems, 34:15032–15043, 2021. [T...
work page internal anchor Pith review arXiv 2021
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.