{"id":"c8e87618-36b8-424a-9579-1e18c10aa5f5","arxiv_id":"2506.20260","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"A bipolar-argumentation framework jointly selects models and counterfactuals so that returned counterfactuals are valid on all selected models, at the cost of majority voting.","lead":"This paper proposes a method that combines model predictions and counterfactual explanations into a single 'argumentative ensemble' so that recourse advice stays valid when several equally good models disagree. A reader might care because it addresses a real gap: under model multiplicity, a counterfactual explanation computed for one model often fails for another equally accurate model.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Theorem 5's robustness guarantee is conditional on every input CE being valid on its own model; if a CE generator returns an invalid CE, the support/attack mirror in Definition 12 breaks and the s-preferred extension can be empty.","rationale":"The paper's theoretical core is sound inside its stated problem: Definition 3 defines a CE as valid on its own model, and Theorem 5's proofs are consistent with that assumption; the experimental use of NICE respects it. The concern is that the abstract and the theorem statement present the guarantee unconditionally, while the guarantee is inherited from an input precondition (valid per-model CEs) that practical CE generators often violate. The single-model counterexample shows the guarantee is not merely weakened but void when a CE is invalid: the s-preferred extension becomes empty. This is the load-bearing point because it affects all four properties in the central claim, not just one. The reader's weakest_assumption identifies exactly this, so I agree. Section 7's explicit limitation statement is honest, which makes the issue addressable rather than fatal; the fix is to state the precondition prominently in the abstract and Theorem 5, and either restrict to validity-guaranteed CE generators or extend the BAF to model invalid CEs. The test-label leakage in Section 6.1.2 is also a real flaw, but it affects the preference-effectiveness experiments, not the central theorem, so it is not the single most load-bearing concern. The verdict remains CONDITIONAL.","tokens_in":27021,"tokens_out":18325,"duration_ms":181483,"concrete_test":"Re-run the Section 6 property evaluation on heloc, compas, and credit using a gradient-based CE generator without validity guarantees (e.g., a Wachter-style method) instead of NICE; count the fraction of inputs for which any model's own CE is invalid, and measure non-emptiness and counterfactual coherence for Sa,s. A nonzero invalidity rate with observed property failures would confirm that the guarantees in Theorem 5 are conditional in a practically relevant way.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central guarantee (Theorem 5) is conditional on the formal definition of a CE in Section 3: each ci must satisfy Mi(ci) != Mi(x). The proofs of counterfactual coherence and non-emptiness depend on the perfect mirror between Mi and ci created by the i=j supports in Definition 12. If a CE generator fails and returns ci with Mi(ci)=Mi(x), Definition 12 still inserts mutual supports (Mi,c1),(c1,Mi) and, because Mi >= Mi, mutual attacks (Mi,c1),(c1,Mi). For a single model M1 with an invalid c1, the only safe set is empty: {M1} set-attacks c1 and also set-supports it, violating safety (Section 3), and {c1} is likewise unsafe; hence P_s contains no nonempty extension, non-emptiness fails, and counterfactual coherence is vacuous. This is not a mere abstraction: Section 6 uses NICE only because it 'guarantees CE validity', while gradient-based CE generators, which the paper cites as common, do not. Section 7 concedes the assumption, but the abstract's 'guarantees the robustness of CEs under MM' and Theorem 5 as stated omit this precondition, so the headline claim is stronger than what is proven.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper formalizes recourse-aware ensembling (RAE) under model multiplicity, introducing six desirable properties for solutions and showing that naive extensions of majority-vote ensembling fail some of them. It then proposes argumentative ensembling, which encodes models and counterfactual explanations as arguments in a bipolar argumentation framework and applies argumentation semantics to select an ensemble. The central theoretical claim is Theorem 5: s-preferred argumentative ensembling satisfies non-emptiness, model agreement, counterfactual validity, and counterfactual coherence, with non-triviality under a stated condition. The paper also claims equivalences between stable and d-preferred semantics and between c-preferred and s-preferred semantics, and reports experiments on three datasets with multiple preference configurations.","tokens_in":27191,"tokens_out":30335,"duration_ms":306733,"significance":"The problem is timely, and the paper's formal decomposition into six properties is a useful contribution regardless of the specific method. The BAF-based approach is original, and the empirical study is broad, with public code and several preference instantiations. However, the main theoretical guarantee, Theorem 5, is false: I provide a small counterexample inside the paper's own definitions in which an s-preferred extension violates counterfactual coherence. Since the paper's contribution rests on this theorem, the theoretical claims as stated cannot be accepted; the empirical results do not compensate for a false central result.","major_comments":[{"comment":"The counterfactual-coherence claim is false, and the proof's step from ⊆-maximality to coherence is invalid. Counterexample with only valid CEs: let M={M1,M2} with M1(x)=M2(x)=0, C={c1,c2}, M1(c1)=1, M2(c2)=1, M2(c1)=0 (so c1 is invalid only on M2), and c2 valid on both models; preferences equal. Under Definition 12, supports are (M1,c1),(c1,M1),(M2,c2),(c2,M2), and the only attacks are (M2,c1),(c1,M2). The set X={M1,c2} is s-admissible: it is conflict-free and safe (M1 supported-attacks M2 via M1→c1→M2, c2 supported-attacks c1 via c2→M2→c1, and no element of X attacks or supports another element of X); it defends M1 against M2's indirect attack M2→c1→M1 by M1→c1→M2, and defends c2 against c1's indirect attack c1→M2→c2 by c2→M2→c1. No superset is s-admissible: adding M2 makes M1 attack M2 while c2 supports M2, and adding c1 makes c2 attack c1 while M1 supports c1. Thus X is an s-preferred extension, yet M1∈X while c1∉X and c2∈X while M2∉X, violating Definition 7. This is not an artifact of invalid CEs: every CE is valid on its own model.","section":"Section 5.3, Theorem 5"},{"comment":"The claimed equivalence between c-preferred and s-preferred argumentative ensembling is false. In the counterexample above, X={M1,c2} is an s-preferred extension but is not closed under the support relation S: M1∈X while c1∉X, and c2∈X while M2∉X. Hence X is not c-admissible. The proof's inference from Theorem 4 and 'Property 2 of [19]' does not establish that every s-admissible set is closed under S; indeed the counterexample shows that s-admissible sets need not be closed. Consequently the paper's claim that c-preferred argumentative ensembling inherits the properties of s-preferred argumentative ensembling is unsupported and, as stated, incorrect.","section":"Section 5.3, Theorem 6"},{"comment":"The headline claim 'guarantees the robustness of CEs under MM' is stated without the precondition that every input CE is valid on its own model. The formal definition of a CE in Section 3 assumes M_i(c_i)≠M_i(x), and Section 7 explicitly concedes that the method assumes a valid CE for each model. If a CE generator returns an invalid CE for its own model, Definition 12 still inserts the i=j support and mutual attack, and the s-preferred extension can be empty, so non-emptiness fails as well. The abstract and theorem statements should carry this precondition explicitly or the method must be extended to handle invalid CEs; as written, the advertised guarantee is stronger than what is proved.","section":"Abstract and Theorem 5"}],"minor_comments":[{"comment":"The proof contains a typographical error: 'Mk(cl)≠Mk(cl)' should presumably read 'Mk(cl)≠Mk(x)'.","section":"Theorem 7 proof"},{"comment":"The implementation deviates from Definition 13: when multiple cardinality-maximal extensions exist, the implementation returns one matching majority vote and, for d-preferred semantics, prefers extensions containing both models and CEs. The reported property-satisfaction rates therefore describe the implemented procedure, not the formal Definition 13 as stated, and this should be acknowledged explicitly.","section":"Section 6.1.4"},{"comment":"Theorem 8 assumes ∀ci∈C, ∀Mi∈M, Mi(ci)≠Mi(x), i.e., every CE is valid on every model, which is substantially stronger than the paper's running assumption that each ci is valid on its own Mi. Section 5.5 motivates the BAF by the possibility of invalid own-model CEs, which is inconsistent with the stated hypothesis of Theorem 8.","section":"Theorem 8"},{"comment":"There are several small proofreading issues: in the definition of A(α1), the set-builder uses '∈X' where it should use '∈A'; in Theorem 5's proof there is a stray 'Definition . 11'; and Table 4 shows identical rows for Sa,d-A and Sa,s-A across all reported entries, which is not explained in the text.","section":"Section 3 and general proofreading"}],"recommendation":"reject","confidential_remarks":"The central theoretical result is false, and the counterexample is small and easy to verify within the paper's own definitions. A correction is not a local repair: it would require either redefining the BAF construction or the solution-selection rule so that s-preferred extensions are genuinely closed under the intended model-CE pairing, or weakening the claimed properties accordingly. The empirical study is useful, but it cannot compensate for a false main theorem in a theory-driven paper. I see no evidence of questionable practice; the result appears to be an overreach in the proof rather than a deliberate misrepresentation."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Worth reading if you care about recourse under model multiplicity. The core argumentative ensembling idea is carried over from their AAMAS 2024 paper, but this version does real new work: it parametrizes four argumentation semantics, proves equivalence results (stable=d-preferred, c-preferred=s-preferred, BAF/AAF), and characterizes property satisfaction with concrete counterexamples. The six RAE properties are sensible, and the observation that naive robust ensembling can return no CEs is a good motivation. The s-preferred guarantee—non-emptiness, model agreement, counterfactual validity, coherence—is plausible and the proofs, while compressed, are checkable. The paper is honest about its ancestry and about the invalid-CE limitation in Section 7.\n\nThe soft spots are real but fixable. First, the abstract says the method 'guarantees the robustness of CEs under MM' without the qualifier that this holds within the selected ensemble and under the assumption that every input CE is valid on its own model. That assumption is built into Definition 12's support relation; if a CE generator returns an invalid CE, the support/attack mirror breaks and even non-emptiness can fail for s-preferred. Section 7 concedes this, but the headline claim is stronger than what is proven. Second, the experiment computes model accuracy preferences on the 20% test inputs that are then used to evaluate the ensembles—that leaks test labels into preference construction and makes the accuracy gains of Sa,s-A unreliable. Third, the code URL has a stray space and doesn't resolve. Fourth, the proofs of Theorems 4 and 6 are noticeably hand-wavy; Theorem 6 in particular jumps from d-preferred=stable to c-admissible closure via a cited property without enough detail. All of these are addressable.\n\nWho is this for? People working on algorithmic recourse, predictive multiplicity, or computational argumentation for XAI. It deserves a serious referee; the theoretical core is sound and the empirical claims, once the preference leakage is fixed, would be informative. I'd send it to review with a request to tighten the abstract, fix the code link, and expand the two proofs.","headline":"Solid incremental extension of the authors' AAMAS 2024 argumentative ensembling framework, with real theoretical content, but the abstract overstates the robustness guarantee and the experiments leak test labels into the preference construction.","tokens_in":27795,"tokens_out":3043,"would_cite":true,"duration_ms":29775,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that under model multiplicity, choosing which models to ensemble and which counterfactual explanations to return must be done together, and proves an argumentation-based method that guarantees every returned…","keywords":["model multiplicity","counterfactual explanations","algorithmic recourse","computational argumentation","bipolar argumentation frameworks","ensembling","argumentative ensembling","recourse-aware ensembling"],"falsifier":"Run s-preferred argumentative ensembling on an instance where one model's own counterfactual is invalid on that model, i.e., $M_i(c_i) = M_i(x)$, and check whether the returned set still contains $M_i$ exactly when it contains $c_i$; the paper's proof of counterfactual coherence depends on the mutual support between $M_i$ and $c_i$, so this case is where the guarantee would fail.","tokens_in":26748,"feed_emoji":"🧩","tokens_out":6919,"duration_ms":66966,"temperature":0.7,"pith_summary":"When several equally accurate models disagree on the same input, a counterfactual explanation computed for one model may be invalid for another, so recourse advice becomes unreliable. This paper formalises the problem of choosing which models to aggregate and which counterfactuals to return at the same time, which it calls recourse-aware ensembling (RAE), and proposes six properties that a good solution should satisfy. It shows that the natural extensions of majority-vote ensembling fail these properties, and it introduces argumentative ensembling, which builds a bipolar argumentation framework in which models attack models that disagree with them and counterfactuals attack models on which they fail. The paper's central claim is that with the s-preferred argumentation semantics, this method guarantees that at least one model and one counterfactual are returned, all returned models agree, every returned counterfactual is valid on every returned model, and a model is returned exactly when its own counterfactual is returned. If true, this gives a principled way to provide robust recourse recommendations under model multiplicity while still letting users express preferences over which models should win.","feed_headline":"Ensembling method guarantees valid recourse across model multiplicity","feed_subtitle":"When equally good models disagree, choosing models and their counterfactuals together keeps every recourse recommendation valid.","key_machinery":"The central object is the Bipolar Argumentation Framework (BAF), a directed graph whose nodes are the models and the counterfactuals, whose attack edges represent conflicts (two models disagreeing on the input prediction, or a counterfactual failing to change a model's prediction), and whose support edges pair each model with its own counterfactual. Model preferences are folded into the attack relation: a less-preferred model can only be attacked, while a more-preferred attacker is always present. Argumentation semantics, specifically the safe-preferred (s-preferred) semantics, which requires the chosen set to be conflict-free, safe, and to defend all its members, select the cardinality-maximal extension, and that extension is the ensemble solution. The support edges are what carry the coherence guarantee: because a model and its counterfactual are supported only by each other, an indirect attack on one is equivalent to a direct attack on the other, which forces the s-preferred extension to contain them together or not at all.","core_discovery":"The paper's central claim is Theorem 5: s-preferred argumentative ensembling satisfies non-emptiness, model agreement, counterfactual validity, and counterfactual coherence, and it satisfies non-triviality under a stated condition. In concrete terms, the ensemble always contains at least one model and one counterfactual, every selected model gives the same prediction for the input, every returned counterfactual flips the prediction of every returned model ($M_i(c_j) \\neq M_i(x)$ for all selected $M_i$ and $c_j$), and a model appears in the solution if and only if the counterfactual computed for that model also appears. The method achieves this by representing each model and each counterfactual as an argument, letting a model attack another model when their predictions differ, letting a counterfactual and a model attack each other when the counterfactual does not flip that model's prediction, and letting each model and its own counterfactual support each other; the s-preferred semantics then selects a maximal set of arguments that is safe and defends itself. The paper also shows that stable semantics coincide with d-preferred semantics and c-preferred with s-preferred under this construction, and it demonstrates empirically on three datasets that the guarantees hold in practice while preferences over model accuracy or simplicity steer the ensemble.","pith_inferences":["The paper assumes every model already has a valid counterfactual; a natural extension the authors leave open is to treat invalid counterfactuals as attackers rather than supporters, which would let the same argumentation machinery repair broken counterfactuals instead of requiring them upfront.","Because the paper proves that stable equals d-preferred and c-preferred equals s-preferred for this construction, the four semantics collapse into two distinct behaviours, so a practitioner only needs to choose between the permissive d-preferred and the strict s-preferred family.","The tie-breaking rule used in the implementation, choosing among cardinality-maximal extensions the one matching majority vote, is an editorial choice not part of the formal definition, so the method's practical behaviour can be better than its theoretical worst case.","The framework's guarantees are binary property checks; a continuous measure of how many models validate a counterfactual, or how close the ensemble is to the majority, would let users trade robustness against majority vote more finely."],"forward_implications":["Under the s-preferred semantics, recourse-aware ensembling always returns a non-empty set of models and a non-empty set of counterfactuals, so the user is never left without a recommendation.","Every counterfactual returned by the s-preferred instantiation is valid for every model in the returned ensemble, which is exactly the robustness under model multiplicity that the paper advertises.","Counterfactual coherence ties each returned counterfactual to its source model: asking for a model's explanation brings the model along, and choosing the model brings its explanation.","User preferences over model properties such as accuracy and simplicity can be injected through the attack relation, and the empirical results show that preference-aware instantiations steer the ensemble toward the preferred property without sacrificing prediction accuracy.","The price of these guarantees is majority vote: argumentative ensembling can return a minority prediction when the majority's counterfactuals are invalid, although empirically the accuracy remains competitive."],"supporting_citations":[{"why":"Supplies the bipolar argumentation semantics (s-admissible, s-preferred, stable) that the argumentative ensembling method relies on.","marker":"[19]"},{"why":"Supplies the abstract argumentation acceptability semantics and the notion of extension used both in the BAF and in the equivalent AAF construction.","marker":"[21]"},{"why":"The authors' prior conference paper that first introduced argumentative ensembling, which this work generalises and re-analyses.","marker":"[20]"},{"why":"Establishes that single-model counterfactuals typically fail across an ensemble, motivating the need for robust recourse under model multiplicity.","marker":"[13]"},{"why":"Provides the model multiplicity framing and the naive ensembling baseline that the paper extends and compares against.","marker":"[1]"},{"why":"Supplies selective ensembling with abstention, a related ensembling baseline that the paper contrasts with its own approach.","marker":"[8]"},{"why":"Shows that counterfactuals on the data manifold are more robust under predictive multiplicity, a related robustness notion the paper builds on.","marker":"[12]"},{"why":"Supplies the nearest-neighbour counterfactual generator (NICE) used in the experiments, which guarantees each counterfactual is valid for its own model.","marker":"[73]"}],"fun_headline_variants":["Argumentative ensembling secures recourse across models","Guaranteed-valid recourse with argumentative ensembling","Robust recourse via argumentative model ensembling","Ensembling argumentation for robust counterfactuals","Argumentative ensembles keep recourse valid under multiplicity"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that every model already has a counterfactual explanation that is valid on that model itself; if any counterfactual generator returns an invalid counterfactual for its own model, the support structure and with it the counterfactual-coherence guarantee break down.","fun_headline_variants_meta":{"raw":{"variants":["Argumentative ensembling secures recourse across models","Guaranteed-valid recourse with argumentative ensembling","Robust recourse via argumentative model ensembling","Ensembling argumentation for robust counterfactuals","Argumentative ensembles keep recourse valid under multiplicity"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000512,"raw_usage":{"total_tokens":2576,"prompt_tokens":1121,"completion_tokens":1455,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":737,"completion_tokens_details":{"reasoning_tokens":1379}},"tokens_in":737,"tokens_out":1455,"duration_ms":12116,"temperature":1.0,"reasoning_tokens":1379,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T22:53:26.902129+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run s-preferred argumentative ensembling on an instance where one model's own counterfactual is invalid on that model, i.e., $M_i(c_i) = M_i(x)$, and check whether the returned set still contains $M_i$ exactly when it contains $c_i$; the paper's proof of counterfactual coherence depends on the mutual support between $M_i$ and $c_i$, so this case is where the guarantee would fail.","supporting_citations":[{"cited_title":"Cayrol, M","cited_arxiv_id":null,"evidence_quote":"Supplies the bipolar argumentation semantics (s-admissible, s-preferred, stable) that the argumentative ensembling method relies on."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the abstract argumentation acceptability semantics and the notion of extension used both in the BAF and in the equivalent AAF construction."},{"cited_title":"Jiang, F","cited_arxiv_id":null,"evidence_quote":"The authors' prior conference paper that first introduced argumentative ensembling, which this work generalises and re-analyses."},{"cited_title":"Leofante, E","cited_arxiv_id":null,"evidence_quote":"Establishes that single-model counterfactuals typically fail across an ensemble, motivating the need for robust recourse under model multiplicity."},{"cited_title":"Black, K","cited_arxiv_id":null,"evidence_quote":"Supplies selective ensembling with abstention, a related ensembling baseline that the paper contrasts with its own approach."},{"cited_title":"Pawelczyk, K","cited_arxiv_id":null,"evidence_quote":"Shows that counterfactuals on the data manifold are more robust under predictive multiplicity, a related robustness notion the paper builds on."},{"cited_title":"Brughmans, P","cited_arxiv_id":null,"evidence_quote":"Supplies the nearest-neighbour counterfactual generator (NICE) used in the experiments, which guarantees each counterfactual is valid for its own model."}],"review_version":1}