{"id":"732cf048-f6ac-439a-bc8a-5e3946d79f6e","arxiv_id":"2412.13762","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":14,"one_line_summary":"An island-based coevolutionary algorithm with mixed Nash equilibrium weighting builds robust decision tree ensembles that outperform prior methods on 20 benchmarks.","lead":"This paper introduces ICoEvoRDF, an evolutionary algorithm that grows decision trees on separate islands and periodically swaps the best ones, then combines them with game-theory weights to resist adversarial attacks. Tests on 20 datasets show it beats existing robust decision tree methods on worst-case accuracy and regret metrics.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Nash-based voting uses mixed-strategy weights as deterministic voting weights; the minimax guarantee applies to a randomized classifier, not the evaluated weighted-vote forest.","rationale":"The reader's weakest assumption concerns whether the internal perturbation populations are a faithful proxy for evaluation-time attacks. That concern is partially addressed by the paper's use of exact MILP adversarial accuracy in evaluation: the MNE weights are tested against worst-case perturbations, not only the evolved ones, and the ablation shows Nash voting improving over equal voting on that exact metric. The more fundamental issue is that the paper's game-theoretic claim is about a mixed strategy over trees, whereas the evaluated model is a deterministic weighted vote over those trees. For a given perturbation P, the accuracy of the weighted vote is not the expected accuracy of a tree drawn from the MNE distribution, so the minimax theorem does not certify the reported ensemble. This mismatch is internal to the paper's argument, not a disagreement with external consensus, and it directly affects the component that lifts ICoEvoRDF above PRAdaBoost in adversarial accuracy. I credit the paper for publishing code, providing extensive ablations, and reporting standard deviations and computation times; those are real strengths. However, the stated MNE contribution needs either a corrected justification, a change of the final model to a randomized classifier, or an empirical demonstration that the deterministic weighted-vote forest realizes or exceeds the game value. Since this is a major but fixable issue, the reader's CONDITIONAL verdict remains appropriate rather than a full rejection. Multiple-comparison correction and the unreported ktop value are secondary concerns and do not change this assessment.","tokens_in":28886,"tokens_out":10743,"duration_ms":107364,"concrete_test":"Using the released ICoEvoRDF code, for each benchmark extract the final island representatives and the reported MNE weights. Build the payoff matrix over these trees and the union of island perturbation populations and compute the game value. Then evaluate three classifiers on the exact MILP adversarial accuracy and the 10^5-perturbation max regret used in the paper: (i) the reported deterministic weighted-vote forest; (ii) a randomized classifier that samples a tree according to the MNE weights, whose expected accuracy is the probability of the correct class under the weight distribution; (iii) the equal-vote forest. If the weighted-vote forest's worst-case payoff over the perturbation populations falls below the game value while the randomized classifier attains it, the MNE-to-voting-weight step is invalid and the gain over equal voting must be re-explained as a heuristic.","verdict_should_be":"UNCHANGED","load_bearing_attack":"In the 'Decision forest composition' section, the authors compute a mixed Nash equilibrium of a zero-sum game whose DT-player pure strategies are one best tree per island, then state that 'probabilities from the mixed equilibrium DT strategy T are directly used for voting weights.' The minimax theorem guarantees the game value only for the randomized classifier that draws a tree according to T: its payoff against perturbation P is E_T[xi(T,P)]. The deterministic weighted-voting forest used for Tables 1-2 has payoff xi(weighted vote, P), which is generally not equal to the expectation over T. Therefore the MNE weights are not justified as voting weights by the game-theoretic argument; the stated theoretical foundation for Nash-based voting is invalid as written. This is load-bearing because the headline empirical advantage over PRAdaBoost in adversarial accuracy depends on the Nash-voting component: in Table 1, ICoEvoRDF with equal voting (ICoEvoRDF^EV) averages 0.779, below PRAdaBoost's 0.782, while the full Nash-weighted variant averages 0.784. The central claim of superior adversarial accuracy therefore rests on a mechanism whose game-theoretic justification does not apply to the reported classifier. The empirical gain may still be real, but it would be a heuristic weighting result, not a direct application of the minimax theorem.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes ICoEvoRDF, an island-based coevolutionary algorithm that evolves decision tree and perturbation populations on multiple islands with periodic migration, and uses a mixed Nash equilibrium to weight trees in the final forest ensemble. The authors claim superior adversarial accuracy and minimax regret over state-of-the-art robust tree ensemble methods across 20 benchmark datasets, and support the claim with ablations, diversity analysis, standard deviations, and a large parameter study on a held-out dataset.","tokens_in":29216,"tokens_out":4695,"duration_ms":41911,"significance":"If the results hold, the main contribution is a flexible island-coevolution framework that can incorporate existing robust tree methods and improve them through migration and game-theoretic weighting. The empirical evaluation is thorough for an evolutionary computation paper: 20 datasets, ablation studies separating the effects of input sampling, migration, and voting, a diversity analysis, standard deviations over 20 runs, and a 10,000-run parameter study on a held-out dataset. The code is publicly available, which is a substantial strength. The principal weakness is that the game-theoretic justification for the Nash-voting mechanism is not valid as stated, and one core migration parameter is never assigned a value in the manuscript, both of which need to be addressed before the central claims can be accepted.","major_comments":[{"comment":"The paper states that 'probabilities from the mixed equilibrium DT strategy T are directly used for voting weights.' The minimax theorem justifies optimality only for a randomized classifier that draws a tree according to T; its payoff is E_T[xi(T,P)]. The deterministic weighted-voting classifier used in Tables 1 and 2 has payoff xi(weighted vote, P), which is not equal to that expectation in general. The game-theoretic foundation for Nash-based voting is therefore not valid as written. This is load-bearing because Table 1 shows the equal-voting variant averaging 0.779, below PRAdaBoost's 0.782, while the Nash-weighted variant averages 0.784; the headline superiority over PRAdaBoost in adversarial accuracy depends on this mechanism. The authors should either provide a rigorous argument for the weighted-vote classifier, or explicitly present the MNE weights as a heuristic weighting scheme and remove the minimax-optimality claim.","section":"Decision forest composition"},{"comment":"The migration count ktop appears in Algorithm 1 (line 6) and in Eq. (5), but no value is given for it in the main text or in the supplementary material. The Parameterization section lists every other hyperparameter (NT, NP, np, Ntop, pc, pm, ps, e, NHoF, lc, lg, ng, |I|) but omits ktop. Since migration is credited with the improvement over independent CoEvoRDT runs in the 'Single decision trees' section, this missing parameter prevents replication and leaves a key mechanism underspecified. Please report the value used in all experiments and, ideally, a sensitivity analysis for ktop.","section":"Parameterization / Algorithm 1"},{"comment":"The improvement of Nash voting over equal voting is not assessed for statistical significance. For adversarial accuracy the average gain is 0.784 - 0.779 = 0.005, which is smaller than the typical reported standard deviation of about 0.010. Given that the central claim relies on the Nash-voting component, the authors should report paired significance tests for the EV vs NV ablation. This would clarify whether the mechanism behind the headline improvement is reliable, especially since only 9 of 17 adversarial-accuracy wins over PRAdaBoost are reported as significant.","section":"Ablation study / Tables 1 and 2"},{"comment":"The max regret results are based on 10^5 randomly sampled perturbations. The standard error analysis in supplementary Table 12 shows that the estimate is stable across different draws, but it does not bound the distance to the true max regret. The claim that ICoEvoRDF outperforms all methods 'in each case with statistical significance' should be qualified as applying to the estimated max regret under this sampling procedure, or the authors should provide evidence that the ranking is robust to the choice of perturbation sample. This limitation should be acknowledged in the main text.","section":"Metrics calculation (supplementary)"}],"minor_comments":[{"comment":"In the supplementary parameterization list, the symbol lc is used both for 'number of consecutive generations for each population' and for 'generations without improvement limit'; in the main text the former is denoted np. This notation conflict should be fixed.","section":"Supplementary, Parameterization"},{"comment":"The main text contains empty cross-references, e.g., 'discussed in Section .' and 'see Section ).' in the 'Islands initialization' and parallelization paragraphs; these should be filled in or removed.","section":"Islands initialization / Results"},{"comment":"The header 'A VERAGE' should read 'AVERAGE', and the ICoEvoRDF variant column headers (ICoEvoRDFEVSI etc.) are cramped; consider splitting the table or using a more readable format.","section":"Table 1"},{"comment":"The statement that existing evolutionary approaches 'operate only with a single population' is imprecise, since CoEvoRDT coevolves a perturbation population alongside the decision tree population; please rephrase to 'single population of decision trees'.","section":"Introduction"},{"comment":"The discussion of parallelization refers to the supplementary pseudocode but does not explain in the main text that the parallel version increments the generation counter by ng/|I|; a brief clarifying sentence would help readers understand the equivalence between the sequential and parallel versions.","section":"Computation time"},{"comment":"The notation N CoEvoRDT is used without a definition in this paper; please define it or provide a pointer to the definition in the cited reference.","section":"Single decision trees"}],"recommendation":"major_revision","confidential_remarks":"The paper fits the journal's scope and the empirical study is unusually solid for an evolutionary computation submission. The main blockers are (1) the invalid game-theoretic justification for the weighted voting as currently written, (2) the missing ktop parameter, and (3) the lack of significance testing for the EV vs NV ablation. These are fixable within the manuscript's scope, so I recommend major revision rather than rejection. The small average advantage over PRAdaBoost in adversarial accuracy (0.784 vs 0.782) makes the significance analysis particularly important."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper is worth reading. It shows that island coevolution with periodic migration improves robust decision trees over independent CoEvoRDT runs, and that a weighted vote derived from a mixed Nash equilibrium of an internal game beats equal voting. The controlled migration-vs-independent-runs comparison is a clean new result, and the ablation study plus diversity analysis back up the mechanism. The empirical case is credible, but the headline gains are modest: average adversarial accuracy is 0.784 versus 0.782 for PRAdaBoost, while the max regret improvement is larger (0.082 versus 0.146).\n\nThe main soft spot is the Nash-voting justification. The authors compute a mixed Nash equilibrium over island representatives and then use the mix probabilities directly as deterministic voting weights. The minimax theorem applies to a randomized classifier that samples a tree according to the mix; the weighted-vote forest has a different payoff function, so the theorem does not justify the weights. The ablation suggests the weighting helps empirically, but that makes it a heuristic, not a direct application of minimax. This needs to be fixed in revision, either by presenting the weights as heuristic or by evaluating the randomized classifier.\n\nSecond, the migration count ktop is never assigned a value in the experimental setup, only listed in pseudocode. That is a genuine reproducibility gap. Third, the significance tests are paired t-tests without multiple-comparison correction, and only 9 of 17 adversarial accuracy wins are individually significant. The max regret metric is estimated from 10^5 random perturbations; the supplementary shows this is stable across draws, so that concern is minor.\n\nThe paper is a solid empirical contribution for people working on robust decision tree ensembles or coevolutionary methods. It deserves a serious referee. I would recommend major revision: fix the Nash-voting claim, report ktop and any other missing hyperparameters, and soften the conclusions to match the effect sizes. With those changes it would be a useful, citable paper.","headline":"Island migration plus MNE weighting is a real, well-ablated empirical improvement over the authors' own CoEvoRDT, but the minimax theorem does not justify the Nash voting weights as used.","tokens_in":29744,"tokens_out":2251,"would_cite":true,"duration_ms":22363,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"An island-based algorithm coevolves trees and attacks per island, migrates top individuals, and weights the final forest by a mixed Nash equilibrium, beating prior robust-tree methods on 20 benchmarks.","keywords":["island model","coevolutionary algorithm","decision forest","adversarial robustness","minimax regret","mixed Nash equilibrium","evolutionary computation","ensemble weighting"],"falsifier":"Compare ICoEvoRDF against its equal-voting ablation on a held-out perturbation set produced by a different attack procedure than the one used during evolution; if equal voting matches or beats Nash voting there, the paper's claimed weighting advantage does not transfer beyond the coevolved attack distribution.","tokens_in":28680,"feed_emoji":"🌲","tokens_out":7745,"duration_ms":66621,"temperature":0.7,"pith_summary":"The paper claims that separating evolution into islands, exchanging top trees between them, and weighting the final forest by a mixed Nash equilibrium of a zero-sum tree-versus-perturbation game produces ensembles that beat existing robust-tree and robust-forest methods on both adversarial accuracy and minimax regret across 20 datasets. The reason to care is practical: decision trees stay interpretable, and this recipe improves their resistance to attacks without giving up that interpretability. The headline evidence is a best average adversarial accuracy of 0.784 and a best average max regret of 0.082 among the tested forest ensembles.","feed_headline":"Island coevolution builds more robust decision forests","feed_subtitle":"Migrating top trees between islands plus Nash-weighted voting lifts robust accuracy to 0.784 and cuts max regret to 0.082.","key_machinery":"The load-bearing mechanism is a zero-sum game between two strategy sets: the fittest decision tree from each island is the tree player's pure strategy, and the union of all island perturbation populations is the perturbation player's pure strategy set. The mixed Nash equilibrium of this game, computed with a standard equilibrium algorithm, supplies the voting weights; in a zero-sum game the minimax theorem guarantees that this weighting maximizes the robustness metric over the strategies considered. Around this, the island model with periodic migration of top trees and perturbations maintains diversity, and distinct per-island training sets sampled with replacement act like a random forest's bagging.","core_discovery":"The central discovery asserted by the paper is that an island model changes the game for coevolutionary robust forests. Instead of one population, multiple islands each coevolve a decision-tree population against a perturbation population, and every ng generations the most fit trees and perturbations migrate between neighboring islands in a ring topology. At the end, the fittest tree from each island enters a zero-sum game against the union of all island perturbation populations, and the mixed Nash equilibrium of that game gives the voting weights. Compared with equal voting, Nash voting adds a small but consistent gain, and compared with repeated independent runs of the single-island method, migration adds a larger gain. The paper reports the best average adversarial accuracy (0.784) and best average max regret (0.082) among the forest methods tested on the 20 benchmarks, and shows that seeding one island with trees from an existing robust-tree algorithm can improve the mixture further.","pith_inferences":["The Nash weights are only as good as the perturbation population that defines the internal game, so a fair stress test is to score the final forest against an attack set generated independently of evolution.","The reported diversity-performance correlation suggests that an explicit diversity-promoting selection pressure or an adaptive migration topology could push the method further, but that goes beyond the paper's experiments.","Since the method's fitness function is a black-box score, the island-plus-MNE recipe can likely be ported to other non-differentiable objectives, including fairness metrics, which the authors list as future work.","Max-regret numbers are computed on a fixed sample of 10^5 random perturbations, so the reported regret gaps should be read as comparisons on that sample; exact adversarial regret would be the ultimate check."],"forward_implications":["Seeding one island with an existing robust-tree method such as FPRDT makes the final forest at least as good as that method and often better, so ICoEvoRDF can act as a wrapper that combines heterogeneous tree generators.","The same machinery produces a single decision tree, and the island version outperforms both the single-population CoEvoRDT and repeated independent runs, so migration is useful even when the final model is one tree.","Because different island training sets and migration increase measured external diversity, and diversity tracks performance, the algorithm's gains should persist with more islands up to a point; the paper finds diminishing returns beyond about 30 islands.","Computation for the island method can be parallelized across islands with a shared memory for migration, cutting wall-clock time by a factor of 8-10 without changing results, which is not available to boosting baselines."],"supporting_citations":[{"why":"The single-island coevolution algorithm whose populations, operators, and Hall of Fame ICoEvoRDF extends, and the baseline it must beat.","marker":"Zychowski, Perrault, and Ma´ndziuk 2024"},{"why":"Source of the robust decision tree baseline RIGDT-h and of the exact MILP-based adversarial accuracy evaluation.","marker":"Chen et al. 2019"},{"why":"Introduces GROOT and the robust random forest baseline that ICoEvoRDF is compared against.","marker":"Vos and Verwer 2021"},{"why":"Introduces FPRDT and PRAdaBoost, the strongest non-evolutionary baselines in the forest comparisons.","marker":"Guo et al. 2022"},{"why":"Establishes the tree-ensemble hardening problem and supplies the MILP method used to compute exact adversarial accuracy.","marker":"Kantchelian, Tygar, and Joseph 2016"},{"why":"Defines max regret, the second objective minimized by ICoEvoRDF.","marker":"Savage 1951"},{"why":"Origin of the island model that motivates periodic migration between subpopulations.","marker":"Tanese 1989"},{"why":"Island-model theory showing migration can improve evolutionary optimization, used to justify the architecture.","marker":"Skolicki and De Jong 2004"},{"why":"Minimax theorem, which guarantees that a mixed Nash equilibrium of the zero-sum tree-versus-perturbation game maximizes the robustness metric.","marker":"v. Neumann 1928"},{"why":"The equilibrium computation algorithm the paper uses to turn mixed Nash equilibrium probabilities into voting weights.","marker":"Lemke and Howson 1964"}],"fun_headline_variants":["Island coevolution and Nash voting harden decision forests","Migrating top trees between islands boosts robustness","Nash-weighted island ensembles beat conventional forests","Island model lifts robust accuracy to 0.784 on benchmarks"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the perturbations evolved on the islands are representative enough of the attacks used to score the final forest, so the Nash voting weights computed on that internal game transfer to evaluation.","fun_headline_variants_meta":{"raw":{"variants":["Island coevolution and Nash voting harden decision forests","Migrating top trees between islands boosts robustness","Nash-weighted island ensembles beat conventional forests","Island model lifts robust accuracy to 0.784 on benchmarks"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000295,"raw_usage":{"total_tokens":1703,"prompt_tokens":926,"completion_tokens":777,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":542,"completion_tokens_details":{"reasoning_tokens":714}},"tokens_in":542,"tokens_out":777,"duration_ms":7790,"temperature":1.0,"reasoning_tokens":714,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T12:49:11.965729+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compare ICoEvoRDF against its equal-voting ablation on a held-out perturbation set produced by a different attack procedure than the one used during evolution; if equal voting matches or beats Nash voting there, the paper's claimed weighting advantage does not transfer beyond the coevolved attack distribution.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Introduces FPRDT and PRAdaBoost, the strongest non-evolutionary baselines in the forest comparisons."},{"cited_title":"D.; and Joseph, A","cited_arxiv_id":null,"evidence_quote":"Establishes the tree-ensemble hardening problem and supplies the MILP method used to compute exact adversarial accuracy."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines max regret, the second objective minimized by ICoEvoRDF."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Origin of the island model that motivates periodic migration between subpopulations."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Island-model theory showing migration can improve evolutionary optimization, used to justify the architecture."},{"cited_title":"E.; and Howson, J","cited_arxiv_id":null,"evidence_quote":"The equilibrium computation algorithm the paper uses to turn mixed Nash equilibrium probabilities into voting weights."}],"review_version":1}