REVIEW 5 major objections 4 minor 28 references
Implementing Ranking-Based Semantics in ConArg: a Preliminary Report
T0 review · 5 major / 4 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read The paper defines a ranking-based semantics that turns Dung labellings into cooperative games and ranks arguments by one of four power indexes, proving the property profile for the Shapley case.
desk verdict A useful but overreaching preliminary report: the Shapley-based ranking is a good idea, but the Deegan-Packel and Johnston formulas are wrong as printed and the acceptance theorems lack valid proofs. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The engine is the pair of characteristic functions $v^I_{\sigma,F}$ and $v^O_{\sigma,F}$ (Definition 9) together with the lexicographic comparison in Definition 10. Instead of introducing a new graph heuristic, the semantics turns the set of extensions of a chosen Dung semantics into a simple voting game, then asks how often and how decisively each argument belongs to winning coalitions. The Shapley value weights marginal contributions by the number of orderings in which a player joins a coalition; Banzhaf counts critical voters without ordering; Deegan-Packel averages over minimal winning coalitions; Johnston gives critical voters a fractional $1/\kappa(S)$ share. The implementation computes $\pi$ only for coalitions $S$ where either $S$ or $S\cup\{i\}$ is an extension, which is exactly where the marginal contribution can be non-zero.
What would settle it
For the AF in Figure 10, compute the Deegan-Packel and Johnston values from the standard definitions (minimal winning coalitions for $\rho$; sum of $1/\kappa(S)$ over winning coalitions for each critical voter for $\gamma$) using the listed extensions, and compare with Tables III and IV: any mismatch shows the implemented indexes are not those named. For Theorem 1, a finite AF giving a violation of any listed property row would refute that row.
Extended reading notes
Core claim
The central claim is that a ranking-based semantics can be defined by taking the in/out labellings of a Dung semantics $\sigma$ as the winning coalitions of a simple game over the arguments: $v^I_{\sigma,F}(S)=1$ exactly when $S$ is an in-set of some $\sigma$-labelling, and $v^O_{\sigma,F}(S)=1$ exactly when $S$ is an out-set of some $\sigma$-labelling. A power index $\pi$ (Shapley, Banzhaf, Deegan-Packel, or Johnston) is then evaluated on these two characteristic functions, and two arguments are compared lexicographically: higher $\pi(v^I_{\sigma,F})$ is better, and in case of a tie, lower $\pi(v^O_{\sigma,F})$ is better. The paper proves, for the Shapley value, that Abstraction, Independence and Totality hold for conflict-free, admissible, complete, preferred and stable semantics; Self-contradiction holds only for conflict-free; Non-attacked Equivalence holds only for complete, preferred and stable; and neither Cardinality Precedence nor Quality Precedence ever holds. It further claims that sceptically accepted arguments receive positive power-index value, credulously rejected arguments receive negative value, and consequently the semantics satisfies $\delta$-Sceptical Precedence and $\delta$-Credulous Precedence for every $\delta$ among the five Dung semantics.
Load-bearing premise
The load-bearing premise is that the Deegan-Packel and Johnston power indexes are faithfully transcribed and implemented (Equations 3 and 4), yet those formulas as written are not well-formed: Equation 3 sums over a set-of-coalitions expression that does not parse, and Equation 4 divides by the critical-voter count of coalitions that do not contain the player, so a mistranscription would make the reported $\rho$ and $\gamma$ rankings something other than the named indexes.
Editorial extensions
If this is right
- For any framework and any of the five Dung semantics, the Shapley-based semantics returns a total ordering of arguments, so no argument is left incomparable.
- Under complete, preferred and stable semantics, all non-attacked arguments receive the same rank; under conflict-free semantics, self-attacking arguments are strictly worse than non-self-attacking ones.
- More numerous or higher-ranked direct attackers do not by themselves make an argument worse: Cardinality Precedence and Quality Precedence fail because an argument's value includes how many other acceptable arguments it defends.
- Sceptically accepted arguments are ranked above merely credulous or rejected arguments, and credulously accepted arguments above rejected ones, for all four power indexes and all five Dung semantics.
- Unlike the Cat, Dbs and Bds ranking semantics, the PI-based semantics satisfies Credulous Precedence for admissible, complete, preferred and stable semantics.
Reading between the lines
- Because $v^I$ and $v^O$ are defined only on exact extension sets, an argument's power-index value is determined entirely by the collection of extensions, not by graph paths; a natural testable extension is to weight coalitions by defence chains or attack distances and see whether Cardinality or Quality Precedence begin to hold.
- Only the Shapley case receives a theorem in this paper; the Banzhaf, Deegan-Packel and Johnston rankings are demonstrated through tables, so a full property profile for those three indexes remains open rather than established.
- The 'out' tie-breaker rewards arguments that are defeated in fewer ways, not just accepted more often, so the semantics can distinguish arguments that look identical when only in-extensions are compared; this could be exploited in frameworks with multiple preferred extensions.
- The implementation's pruning of coalitions to those that are extensions may avoid the generic NP-hardness of power-index computation, and combining the Shapley value with restricted-coalition game values would let the ranking ignore arguments that are not even credulously accepted.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a ranking-based semantics for abstract argumentation, called PI-based semantics, in which each argument is ranked by the lexicographic comparison of two power-index values computed from characteristic functions vI and vO that mark, respectively, the in-labellings and out-labellings of a selected Dung semantics. The authors claim to instantiate this parametric semantics with four cooperative-game power indexes (Shapley, Banzhaf, Deegan-Packel, Johnston), state an axiom profile for the Shapley instance (Theorem 1), state theorems connecting power-index sign with credulous/sceptical acceptance (Theorem 2 and Proposition 1), describe an implementation inside the ConArg web tool, and illustrate the rankings on a worked example.
Significance. The underlying idea is appealing and potentially useful: it connects ranking-based argumentation with well-studied cooperative-game power indexes and makes classical Dung semantics a parameter of the ranking. The Shapley and Banzhaf columns in Tables I and II are plausible and suggest that the tool works for those indexes. If the formal claims were properly proved and all four indexes were implemented faithfully, the paper would be a useful bridge between computational social choice and abstract argumentation. However, the manuscript as submitted does not support the claims for Deegan-Packel and Johnston: the defining equations are not mathematically well-formed, the reported numerical values are inconsistent with the named indexes, and Theorem 2 is contradicted by the paper's own Table III. No machine-checked proofs, reproducible code, or independent verification is provided for the property theorems.
major comments (5)
- [Section II-B, Eq. (3)] The Deegan-Packel formula as printed is not well-formed: the summation ranges over S⊆M_i(v)\setminus{i}, but M_i(v) is a family of coalitions, so the subtraction of the player i from this family is undefined and S would range over subsets of a set of coalitions rather than over coalitions. The standard Deegan-Packel index is ρ_i(v) = (1/|M(v)|) Σ_{S∈M_i(v)} 1/|S|, with no marginal term v_{Si}. The values reported in Table III (for example, the ρ-COM row 0.5, 0, 0.5, 0, 0) are consistent with the standard formula using M(v) = {{a,c}} but do not follow from Eq. (3) as written. The paper must correct Eq. (3), state which definition is actually implemented, and reconcile the implementation note in Section IV-A, where the computation is described in terms of marginal contributions v(S∪{i})−v(S), a construction appropriate for Shapley and Banzhaf but not for Deegan-Packel.
- [Section II-B, Eq. (4) and Table IV] The Johnston index is mis-defined. The standard Johnston index credits player i only in winning coalitions that contain i, dividing by the number of critical voters in that coalition, so it is nonnegative. Equation (4), by contrast, sums marginal contributions over S⊆N\setminus{i} and divides by κ(S), where κ(S) is the number of critical voters in a coalition that does not contain i. The implementation note in Section IV-A confirms that the tool computes v(S∪{i})−v(S) for all four indexes, and Table IV reports impossible negative Johnston values, e.g., γ-CF v_I(b) = −3.16667 and γ-ADM v_I(b) = −6.16667. Therefore the γ rankings and all γ-related property claims in Sections III and V are not about the Johnston index as conventionally defined. Both the equation and the implementation must be corrected before the parametric-family claim can be assessed.
- [Section III, Theorem 2] Theorem 2 is false for the Deegan-Packel index under the standard (nonnegative) definition. If an argument is never accepted, it belongs to no minimal winning coalition, so its Deegan-Packel value is 0, not strictly negative as the theorem claims. This is visible in the paper's own Table III: the ρ-COM row gives b the value 0.00000 even though b is never in a complete extension. The statement 'if a is credulously rejected then π_a(v_I) < 0' can hold at best for the Shapley and Banzhaf instances, and the theorem should be re-stated index by index after the Deegan-Packel and Johnston definitions are fixed.
- [Section III, Proposition 1 and its proof] The proof of Proposition 1 is not a valid derivation. From the fact that a credulously accepted argument i has at least one positive marginal contribution, it does not follow that the aggregated index value of i is higher than that of a rejected argument j; the aggregate is a weighted sum over all coalitions and may include many negative terms. A correct proof must compare the full weighted sums, or restrict to indexes for which the comparison can be established. The same gap affects the 'straightforward' proof of Theorem 2. Given that these results are presented as theorems and propositions, the authors need to supply complete proofs from the actual definitions.
- [Section III, Theorem 1] The proof of Theorem 1 is informal and figure-based, and it is not possible to check the claimed property profile from the text. For example, the Self-contradiction case asserts an inequality involving E and v_I without deriving the Shapley comparison, and the other cases are delegated to Figures 2–6 with no numerical evaluations. Since Theorem 1 is the main theoretical contribution, the authors should provide either formal derivations or a machine-checkable verification script that reproduces each claim. If the paper is intended strictly as a tool report, the property statements should be presented as experimental findings rather than as proven theorems.
minor comments (4)
- [Abstract and Introduction] The phrase 'well know properties' should be 'well-known properties'.
- [Definition 4] The definition of a strict ranking contains a typo: 'a ≻F b is a shortcut for a ≽F b and b⁄≽F b' should read 'b⁄≽F a'.
- [Table III] The entries '0, 16667' use a decimal comma instead of a decimal point, which is inconsistent with the other tables and should be corrected.
- [Section II-B] The citation for the Johnston index is imprecise: the text refers to Johnston via reference [21], a general encyclopedia, while the formal definition is attributed to the Durán et al. paper [18]; the citation should be aligned with the actual source of the formula.
Circularity Check
No circularity: the PI-based semantics is explicitly defined in Definitions 9 and 10, and the claimed properties are derived from that definition and standard Dung-semantics facts; no fitted parameter is renamed as a prediction.
full rationale
The paper's load-bearing content is a definition of a ranking-based semantics in terms of power indexes over characteristic functions built from Dung extensions, followed by property statements proven from that definition. The core ranking is not fitted to the reported rankings or property outcomes; Tables I–IV are illustrative outputs of the implemented formulas, not predictions used to define the semantics. The tie-breaking use of vO is an explicit definitional choice, not an input disguised as a result. The self-citations to [6] and [7] are contextual (preliminary ideas and tool implementation) and do not carry the proof burden: Definition 10 is given in this paper and the proofs are self-contained, albeit concise. The potentially malformed Deegan-Packel and Johnston equations (Eqs. 3 and 4) and the negative Johnston values in Table IV are correctness or implementation-fidelity concerns about whether the tool computes the standard indexes; they are not circularity, because no claim reduces to its own input by construction. The abbreviated proof of Theorem 2 is an expositional omission rather than a circular step. Overall, no load-bearing derivation collapses into its inputs, so the circularity score is 0.
Assumptions & free parameters
assumptions (4)
- domain assumption The in-sets of a Dung semantics σ define the winning coalitions of a simple game
- domain assumption Standard power-index formulas (Shapley, Banzhaf, Deegan-Packel, Johnston) apply unchanged to the extension game
- ad hoc to paper Lexicographic tie-breaking through vO gives a meaningful refinement of the ranking
- ad hoc to paper The empty set is treated as a (minimal) winning coalition for Deegan-Packel
invented entities (2)
-
PI-based ranking semantics
-
Characteristic functions vI and vO
Cite this review
Pith. "Pith review of Implementing Ranking-Based Semantics in ConArg: a Preliminary Report." pith.science (2026). https://pith.science/paper/BKO6A5L2
@misc{pith2026190807784,
author = {Pith},
title = {Pith review of: Implementing Ranking-Based Semantics in ConArg: a Preliminary Report},
year = {2026},
howpublished = {\url{https://pith.science/paper/BKO6A5L2}},
note = {Machine review of arXiv:1908.07784}
}
read the original abstract
ConArg is a suite of tools that offers a wide series of applications for dealing with argumentation problems. In this work, we present the advances we made in implementing a ranking-based semantics, based on computational choice power indexes, within ConArg. Such kind of semantics represents a method for sorting the arguments of an abstract argumentation framework, according to some preference relation. The ranking-based semantics we implement relies on Shapley, Banzhaf, Deegan-Packel and Johnston power index, transferring well know properties from computational social choice to argumentation framework ranking-based semantics.
Figures
Figures from the paper (5 more)
Reference graph
Works this paper leans on
-
[1]
J. M. Alonso-Meijide, M. ´Alvarez-Mozos, and M. G. Fiestras-Janeiro. Power Indices and Minimal Winning Coalitions in Simple Games with Externalities. UB Economics Working Papers E15/328, University of Barcelona - Department of Actuarial, Financial and Economic Mathe- matics, 2015
work page 2015
-
[2]
L. Amgoud and J. Ben-Naim. Ranking-Based Semantics for Argumen- tation Frameworks. In Proc. SUM 2013 - 7th International Conference , volume 8078 of LNCS, pages 134–147. Springer, 2013
work page 2013
-
[3]
J. F. Banzhaf. Weighted V oting Doesn’t Work: A Mathematical Analysis. Rutgers Law Rev , 19(2):317–343, 1965
work page 1965
- [4]
-
[5]
P. Besnard and A. Hunter. A logic-based theory of deductive arguments. Artif Intell, 128(1-2):203–235, 2001
work page 2001
-
[6]
S. Bistarelli, F. Faloci, F. Santini, and C. Taticchi. A Tool For Ranking Arguments Through V oting-Games Power Indexes. In Proceedings 34th CILC, volume 2396 of CEUR Workshop Proceedings , pages 193–201. CEUR-WS.org, 2019
work page 2019
-
[7]
S. Bistarelli, P. Giuliodori, F. Santini, and C. Taticchi. A Cooperative- game Approach to Share Acceptability and Rank Arguments. In Proceedings of the 2nd Workshop on Advances In Argumentation In Artificial Intelligence, AI3@AI*IA , volume 2296 of CEUR Workshop Proceedings, pages 86–90. CEUR-WS.org, 2018
work page 2018
-
[8]
S. Bistarelli, T. Mantadelis, F. Santini, and C. Taticchi. Probabilistic Argumentation Frameworks with MetaProbLog and ConArg. In Pro- ceedings of IEEE 30th International Conference on Tools with Artificial Intelligence, ICTAI, pages 675–679. IEEE, 2018
work page 2018
Show all 28 references
-
[9]
Bistarelli, F
S. Bistarelli, F. Rossi, and F. Santini. ConArg: A Tool for Classical and Weighted Argumentation. In Computational Models of Argument - Pro- ceedings of COMMA , volume 287 of Frontiers in Artificial Intelligence and Applications, pages 463–464. IOS Press, 2016
2016
-
[10]
Bistarelli and F
S. Bistarelli and F. Santini. ConArg: A Constraint-Based Computational Framework for Argumentation Systems. In Proceedings of IEEE 23rd International Conference on Tools with Artificial Intelligence, ICTAI , pages 605–612. IEEE Computer Society, 2011
2011
-
[11]
Bonzon, J
E. Bonzon, J. Delobelle, S. Konieczny, and N. Maudet. A Comparative Study of Ranking-Based Semantics for Abstract Argumentation. In Proc. Thirtieth AAAI Conference on Artificial Intelligence , pages 914–920. AAAI Press, 2016
2016
-
[12]
Caminada
M. Caminada. On the Issue of Reinstatement in Argumentation. In Lect Notes Artif Int, 10th European Conference, JELIA , volume 4160 of LNCS, pages 111–123. Springer, 2006
2006
-
[13]
ˇCyras, D
K. ˇCyras, D. Letsios, R. Misener, and F. Toni. Argumentation for explainable scheduling. In Proc. Thirty-Third AAAI Conference on Artificial Intelligence, volume 33, pages 2752–2759. AAAI Press, 2019
2019
-
[14]
Derks and H
J. Derks and H. Peters. A Shapley Value for Games with Restricted Coalitions. Int J Game Theory , 21(4):351–60, 1993
1993
-
[15]
P. Dondio. Ranking Semantics Based on Subgraphs Analysis. In Proc. 17th AAMAS International Conference , pages 1132–1140. IFAAMAS Richland, SC, USA / ACM, 2018
2018
-
[16]
P. M. Dung. On the Acceptability of Arguments and its Fundamental Role in Nonmonotonic Reasoning, Logic Programming and n-Person Games. Artif Intell, 77(2):321–358, 1995
1995
-
[17]
P. E. Dunne and M. J. Wooldridge. Complexity of abstract argumenta- tion. In Argumentation in Artificial Intelligence, pages 85–104. Springer, 2009
2009
-
[18]
E. A. Dur ´an, J. M. Bilbao, J. R. F. Garc ´ıa, and J. J. L ´opez. Computing power indices in weighted multiple majority games. Math Soc Sci , 46(1):63–80, 2003
2003
-
[19]
Gabbriellini and P
S. Gabbriellini and P. Torroni. Abstract argumentation for agent-based social simulations. In Proc. 10th ArgMAS International Workshop, pages 1–14, 2013
2013
-
[20]
Grossi and S
D. Grossi and S. Modgil. On the Graded Acceptability of Arguments. In Proc. IJCAI-15, pages 868–874. AAAI Press, 2015
2015
-
[21]
D. Keith. Encyclopedia of Power. SAGE Publications, Inc., 2011
2011
-
[22]
the story of the poor public good index
S. Kurz. Which criteria qualify power indices for applications? - A comment to “the story of the poor public good index”. CoRR, abs/1812.05808, 2018
2018 arXiv
-
[23]
Leite and J
J. Leite and J. Martins. Social Abstract Argumentation. In Proc. IJCAI- 11, pages 2287–2292. IJCAI/AAAI, 2011
2011
-
[24]
Longo, B
L. Longo, B. Kane, and L. Hederman. Argumentation theory in health care. In Proc. CBMS 2012, 25th IEEE International Symposium , pages 1–6. IEEE Computer Society, 2012
2012
-
[25]
Matsui and Y
T. Matsui and Y . Matsui. A survey of algorithms for calculating power indices of weighted majority games. J Oper Res Soc Jpn , 43:71–86, 2000
2000
-
[26]
Matt and F
P. Matt and F. Toni. A Game-Theoretic Measure of Argument Strength for Abstract Argumentation. In Lec Notes Artif Int, 11th European Conference, JELIA , volume 5293 of LNCS, pages 285–297. Springer, 2008
2008
-
[27]
L. S. Shapley. Contributions to the Theory of Games , volume II of AM-28. Princeton University Press, 1953
1953
-
[28]
E. Winter. The shapley value. In Handbook of Game Theory with Economic Applications, volume 3, pages 2025–2054. Elsevier, 2002
2025
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.