{"id":"85e26368-5057-4b02-96f9-ce8b21b973cc","arxiv_id":"2501.07515","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":2.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"A survey argues that bioinspired optimization suffers from a 'paradox of success' in which many metaphor-based algorithms lack real novelty, and lays out methodological remedies.","lead":"This paper reviews a growing pile of criticism against evolutionary and bioinspired optimization algorithms: too many new 'nature-inspired' methods add little beyond a new metaphor, and many experiments are not rigorous. It organizes the main critiques and collects proposed fixes, from better benchmarks and statistical tests to automated algorithm design.","discovery_kind":"review","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 'more than half incremental' claim rests on the authors' own unvalidated taxonomy; without independent sampling, the systemic diagnosis may overstate prevalence.","rationale":"The paper is a well-organized synthesis of existing critiques and methodological proposals in evolutionary and bioinspired optimization. It does not contain an internal technical contradiction, and the existence of many non-innovative metaphor-based algorithms is independently supported by the literature it cites (e.g., Weyland [29], Piotrowski et al. [30], Camacho-Villalón et al. [53,57], Kudela [61]). The reader's CONDITIONAL verdict is appropriate. My stress-test pass identifies the same weakest point as the reader: the central claim that a 'large share' of the field is non-innovative depends on the representativeness of the curated critical literature and, in particular, on the quantitative estimate from the authors' own taxonomy [19] that 'more than half' of reviewed proposals are incremental variants of PSO, DE, and GA. That estimate is not independently verified in this paper, and the selected examples in Sections 2.1 and 3.1 are not a random sample. The proposed methodological pathways (equation-level equivalence, configuration-level equivalence, automated design) are plausible and grounded in prior work, but their effectiveness as a remedy is asserted rather than empirically demonstrated. Neither concern is fatal to the paper's value as a position statement and reference point, but both justify the conditional framing. I therefore recommend no change to the reader's verdict, while noting that the quantitative prevalence claim should be treated as a hypothesis requiring independent confirmation rather than an established fact.","tokens_in":21032,"tokens_out":3402,"duration_ms":34107,"concrete_test":"Take a random sample of 200 algorithm-proposal papers published 2015–2024 in venues such as Swarm and Evolutionary Computation, Applied Soft Computing, and Information Sciences (excluding the authors' own taxonomy's selected cases). Have two independent raters apply the taxonomy definitions from [19] and the equivalence methodology in Section 3.2 (e.g., using the operator decomposition of de Armas et al. [72]) to classify each proposal as incremental/homologous vs. innovative. If inter-rater agreement is low, or if the proportion of incremental proposals is not significantly above 50%, the paper's central quantitative claim fails. A secondary check: apply the proposed pathway to a known set of 10 algorithms (5 criticized, 5 considered innovative) and verify it reproduces expert classifications; if it does not, the remedy is unvalidated.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 2.1's pivotal quantitative claim—'more than half of the proposals reviewed in our work are incremental, minor versions of only three very classical algorithms'—is quoted from the authors' own taxonomy [19] but is not accompanied in this paper by inter-rater reliability, independent replication, or a clear operational definition of 'incremental, minor version.' The examples in Sections 2.1 and 3.1 (Harmony Search, Black Hole Optimization, Grey Wolf Optimizer, Raven Roost Optimization, etc.) are selected critical case studies; they show that such weak proposals exist, but do not establish that they constitute more than half of the literature. If the true proportion were much lower (e.g., 20–30%), the 'paradox of success' would be a mischaracterization of a minority pathology, and the paper's call for sweeping editorial policy changes in Section 2.1 would be disproportionate. The pathways in Section 3.2 are offered as the remedy, but their effectiveness is asserted rather than demonstrated; the paper presents no empirical case where equation-level or configuration-level equivalence screening changed acceptance outcomes or improved algorithm design. Thus the most load-bearing assumption is the representativeness of the critical literature and taxonomy, on which both the diagnosis and the remedy rest.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This manuscript is a review/position paper on methodological rigor in evolutionary and bioinspired optimization. It argues that the field exhibits a 'paradox of success': the large and growing volume of publications proposing nature- or bio-inspired solvers has not produced commensurate algorithmic innovation. Section 2 organizes known critiques into three categories: lack of algorithmic novelty, low-quality experimental comparisons, and poor or biased benchmarks. Sections 3-5 present pathways for detecting weak proposals (equation-level and configuration-level equivalence tests), for fair and replicable benchmarking, and for improving existing solvers. Section 6 discusses automated and LLM-based algorithm design as a promising direction. The authors conclude by urging researchers, reviewers, and editors to adopt the proposed guidelines. The paper's contribution is synthesis and normative guidance rather than new empirical evidence.","tokens_in":21143,"tokens_out":9161,"duration_ms":85243,"significance":"If accepted as a synthesis, the paper is a useful and well-organized digest of a dispersed critical literature. Its strengths include accurate quotation and attribution of key critiques, explicit translation of those critiques into concrete equivalence and benchmarking procedures, and practical recommendations for authors and editors. It also points to immediate tools such as ablation tests, origin-bias tests, and formal verification. The main limitation is that the paper does not itself establish the prevalence of weak proposals: the 'more than half' statistic is inherited from the authors' prior taxonomy, and the proposed pathways are not evaluated against outcome data. These limits affect the strength of the systemic diagnosis but do not negate the value of the review as a roadmap.","major_comments":[{"comment":"The abstract and Section 2.1 treat the claim that 'more than half of the proposals reviewed in our work are incremental, minor versions of only three very classical algorithms' as a factual characterization of the field. This statistic is quoted from the authors' own taxonomy [19], but the present manuscript does not report the operational definition of 'incremental, minor version,' the inclusion criteria of the reviewed corpus, inter-rater reliability of the classification, or any independent replication. Since this proportion is the empirical backbone of the 'paradox of success' and motivates the call for editorial policy changes, the authors should either provide additional evidence for the representativeness of this estimate or explicitly restate it as a hypothesis/estimate from one classification study, with a discussion of how selection bias in that corpus could change the conclusion.","section":"Section 2.1, quoted passage from [19]"},{"comment":"The critical examples (Harmony Search, Black Hole Optimization, Grey Wolf Optimizer, Raven Roost Optimization, and others) are presented as evidence of weak proposals, but the paper does not describe how these particular studies were selected from the wider critical literature. No systematic search strategy, inclusion/exclusion criteria, or temporal coverage is given. Without this, the reader cannot distinguish a systemic, field-wide phenomenon from a list of deliberately chosen pathologies. The authors should add a short methodology paragraph or a table of all analyzed critical studies with their targets and methods so that the representativeness of the synthesis can be assessed.","section":"Sections 2.1 and 3.1, selection of critical case studies"},{"comment":"The paper presents equation-level equivalence, configuration-level equivalence, and automated design as 'methodological pathways' that will refocus the field, and Section 7 states that 'the analyzed pathways are designed to refocus optimization research on its ultimate goal.' Yet no evidence is offered that applying these pathways changes editorial decisions, prevents publication of weak solvers, or improves algorithm design. For instance, no case is shown where an equivalence screen detected a non-innovative algorithm before publication, and the LLM-based systems in Section 6.2 are described without an evaluation of whether their outputs satisfy the equivalence and benchmarking criteria advocated earlier. The pathways may be reasonable prescriptions, but the manuscript should either soften these claims to 'proposals to be validated' or cite pilot studies that demonstrate their efficacy.","section":"Sections 3.2 and 6, proposed pathways as remedies"}],"minor_comments":[{"comment":"The sentence beginning 'However, this is not the case in many of such studies. as underscored in [19]:' has a punctuation error; 'as' should not begin a new sentence after the period.","section":"Section 2.2"},{"comment":"The sentence 'A comprehensive component-based analysis of each algorithm to substantiate this assertion, these algorithms are identified as variants of Particle Swarm Optimization and Evolution Strategies.' is grammatically broken and should be rewritten.","section":"Section 3.1"},{"comment":"The phrase 'straw-grain discrimination studies' is unclear; consider 'wheat-from-chaff discrimination' or 'differentiating weak from strong proposals.'","section":"Section 3.2"},{"comment":"Reference [13] contains a malformed DOI ('https://doi.org/10.1109/10.1016/j.asoc.2023.110757'); the DOI should be corrected or the URL cleaned.","section":"References"},{"comment":"The paper reports inherited quantitative claims ('more than half' in Section 2.1 and '65%' in Section 5) without a summary table; adding a table with the source, method, and study corpus for each quantitative claim would make the evidence easier to verify.","section":"Sections 2.1 and 5"},{"comment":"The statement that 'almost a hundred important researchers' signed the open letter [24] should be updated with the exact number or a citation to a source that tracks the count, since the number may have changed since the letter's initial publication.","section":"Section 2.1"}],"recommendation":"major_revision","confidential_remarks":"The manuscript fits the scope of a methodological review journal. The most important issue for the editor is the provenance of the 'more than half' estimate: it comes from the authors' own prior taxonomy and is repeated as a field-level fact. This is not misconduct, but the paper should be explicit about the self-referential nature of this evidence. I also note the high number of self-citations (e.g., [19], [26], [37], [49], [65], [66]); most are relevant, but the reliance on them in the central argument should be disclosed. No other concerns."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear colleague,\n\nYou can treat this as a competent overview of a debate that already exists elsewhere. The paper's own contributions are the organization of known critiques into 'pathways' — equation-level equivalence, configuration-level equivalence, automated design — plus a short section on LLM-generated algorithms. There is no new algorithm, dataset, or proof.\n\nWhat it does well: it accurately reproduces the key critical literature. Weyland on Harmony Search, Piotrowski on Black Hole Optimization, Camacho-Villalón on the GWO/Firefly/Bat family, Kudela's bias studies, the Bestiary, and the open letter are all represented fairly. The structure is clear, and Section 3.2 is a reasonable synthesis of proposals that individuals have floated. The benchmarking discussion (center bias, representativeness, replicability) is solid and well-sourced.\n\nThe soft spots are about proportion and evidence. The load-bearing statistic — that more than half of proposals are incremental minor variants of PSO, DE, and GA — comes from the authors' own taxonomy [19]. This paper provides no inter-rater reliability, no operational definition of 'incremental minor version,' and no independent sampling. The selected examples show that weak proposals exist, but they do not establish that most of the literature is weak. If the true share is lower, the 'paradox of success' reads as a significant minority pathology rather than a systemic property. The proposed pathways are plausible, but the paper does not demonstrate that adopting them improves outcomes or changes editorial decisions. That is a minor issue for a position paper, but worth saying.\n\nThe self-citation pattern is noticeable — the authors lean heavily on their own taxonomy and guidelines — but the core criticisms are grounded in external work, so it is not a load-bearing defect.\n\nWho is this for? Editors, reviewers, and newcomers who want one entry point into this critique landscape. It is a useful reference, not a research advance. It deserves a serious referee: the topic matters to the field, and the authors clearly know the material. I would send it to review, asking the authors to either soften the 'more than half' claim or support it with a transparent sampling methodology, and to flag which recommendations rest on their own prior work versus independent consensus.","headline":"A fair, well-organized synthesis of known critiques; the 'more than half' claim is the soft spot, but the paper earns a review.","tokens_in":21783,"tokens_out":2811,"would_cite":true,"duration_ms":26961,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["68W50","90C59"],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper argues that bioinspired optimization's publishing boom masks a core crisis: most new solvers are minor variants of three classical algorithms.","keywords":["Bioinspired computation","Evolutionary computation","Metaheuristics","Methodological critique","Benchmarking","Algorithmic innovation","Metaphor-based algorithms","Automated algorithm design"],"falsifier":"A random or systematic audit of a sizable sample of published bioinspired algorithms—formalizing each solver's search operators and testing equation-level and configuration-level equivalence against modern PSO, DE, and GA—that found the majority are not equivalent to those three classical families would refute the field-level claim, though individual case studies like Harmony Search or Black Hole Optimization could still stand.","tokens_in":20732,"feed_emoji":"🧬","tokens_out":9561,"duration_ms":82167,"temperature":0.7,"pith_summary":"This paper tries to establish that evolutionary and bioinspired optimization are in a 'paradox of success': publication volume has grown enormously, but the field's core is weakened by inadequate benchmarking, problem-specific overfitting, thin theoretical grounding, and proposals whose only justification is a biological metaphor. It argues, citing its own taxonomy, that more than half of the reviewed nature-inspired solvers are incremental, minor versions of just three classical algorithms—Particle Swarm Optimization, Differential Evolution, and Genetic Algorithms. The review recasts the accumulated critical literature as a constructive diagnosis and collects the methodological pathways proposed to fix it: detecting weak proposals through equation-level and configuration-level equivalence, fair and replicable benchmarking, real-world-relevant test problems, and automated algorithm design. A sympathetic reader should care because the paper is trying to say that the field can keep its momentum only by making novelty and experimental rigor the explicit standard for publication. If the diagnosis is right, a large share of published solvers are not contributions but recycled designs, and remediation is a community-level editorial and methodological project rather than a technical tweak.","feed_headline":"Most 'novel' bioinspired solvers are recycled PSO, DE, GA","feed_subtitle":"A critical review argues the publishing boom hides weak benchmarking and metaphor-only novelty, and maps fixes.","key_machinery":"The analytical engine is a two-layer recapitulation. The first layer is the taxonomy of [19], which classifies solvers by inspiration versus algorithmic behavior and supplies the quantitative claim that most proposals are incremental variants of PSO, DE, and GA. The second layer is the pair of equivalence criteria proposed for separating 'wheat from chaff': equation-level equivalence, which compares search operators mathematically at the operator or component level (through homologous-component studies, formal verification, operator simplification, and ablation tests), and configuration-level equivalence, which asks whether parameter settings make a new solver behave essentially like an existing one. These criteria do the work of turning the critical literature into an actionable test: a proposal that is equation- or configuration-equivalent to an existing solver is non-innovative, one that generalizes an existing solver makes the old version redundant, and only a solver whose behavior cannot be replicated by adjusting parameters counts as genuinely innovative. The review also presents benchmarking and replicability standards, and automated design (with and without large language models) as the constructive forward path.","core_discovery":"The paper's central claim, stated in the abstract and developed through the review, is that both areas are 'plagued by challenges at their core': a lack of algorithmic innovation, low-quality experimental studies, and poor benchmarks, with a steady flow of 'superfluous proposals justified only by their biological metaphor.' Section 2.1 commits the paper to the stronger empirical estimate from its own taxonomy that more than half of the proposals reviewed are incremental, minor versions of only three very classical algorithms (Particle Swarm Optimization, Differential Evolution, and Genetic Algorithms). The review then assembles the case-study literature exposing individual weak proposals—Harmony Search as essentially a special case of Evolution Strategies, Black Hole Optimization as a simplification of Particle Swarm Optimization, and Grey Wolf, Firefly, and Bat algorithms as reformulations of existing PSO variants—and converts these critiques into positive pathways for detecting equivalence, benchmarking fairly, and automating algorithm design. It is not an experimental demonstration; it is an argument that the field's success in volume has not produced corresponding innovation, and that the proposed pathways are the right remedy.","pith_inferences":["A testable extension the paper leaves implicit is to run its equivalence-detection machinery over a large random sample of the 500+ solvers already classified in its taxonomy and publish the distribution of equivalence classes; that would calibrate the 'more than half' estimate with formal methods rather than expert judgment.","The configuration-level criterion implies a stronger practical corollary: for every allegedly novel solver, some parameterized portfolio of the classical algorithms could match its behavior across a benchmark suite, so the practical contribution of many published solvers would reduce to tuning rather than search logic.","The same audit could be adapted to neighboring publication-heavy fields in machine learning where novelty claims are name- or metaphor-driven, although the equivalence formalisms would need to be reworked beyond population-based search.","A community-level testable design extension is to make automated-design software frameworks the default venue for proposing solvers, so that novelty is demonstrated by verifiable performance on real-world problems rather than by a new metaphor."],"forward_implications":["If the taxonomy's estimate holds, any new bioinspired solver should be compared against modern, well-tuned versions of Particle Swarm Optimization, Differential Evolution, and Genetic Algorithms rather than naive classical baselines, because beating the naive versions is easy and proves little.","Equation-level and configuration-level equivalence tests give reviewers and editors a concrete procedure to classify a proposal as non-innovative, making the open letter's editorial demands operational.","Because many popular solvers carry a bias toward the center of the search domain, experiments on shifted and diverse benchmark functions are necessary to avoid reporting a bias artifact as a performance gain.","Replicability requirements such as extensible templates, white-box problem descriptions, and remotely accessible frameworks would make published results externally verifiable, weakening the publish-or-perish incentives behind metaphor-based method proposals.","If automated design, including LLM-driven generation, becomes the norm, the biological metaphor stops being the criterion of novelty and the field's value shifts toward solving real-world problems, which the paper states as its ultimate goal."],"supporting_citations":[{"why":"Supplies the paper's quantitative core: more than half of reviewed nature-inspired solvers are minor variants of PSO, DE, and GA.","marker":"[19]"},{"why":"Foundational critique that metaphor alone does not make a metaheuristic novel, framing the whole diagnosis.","marker":"[21]"},{"why":"Documents the tide of metaphor-based method proposals and the publish-or-perish incentives behind them.","marker":"[23]"},{"why":"Open letter signed by field experts demanding editorial rejection of non-innovative metaphor-based solvers, giving the paper its call to action.","marker":"[24]"},{"why":"Provides the methodological comparison guidelines the paper presents as the path to fair experiments.","marker":"[26]"},{"why":"Argues automated design can replace manual metaphor-driven design, grounding the paper's forward-looking pathway.","marker":"[27]"},{"why":"Early case study showing Harmony Search is essentially a special case of Evolution Strategies, a canonical weak-proposal example.","marker":"[29]"},{"why":"Shows Black Hole Optimization is a simplification of Particle Swarm Optimization, another canonical weak-proposal example.","marker":"[30]"},{"why":"Exposes the center-bias benchmarking problem and proposes a shared repository of real-world problems, supporting the benchmarking critique.","marker":"[42]"},{"why":"Provides component-level evidence that Grey Wolf, Firefly, and Bat algorithms are reformulations of PSO variants.","marker":"[53]"}],"fun_headline_variants":["Many 'novel' optimizers are just PSO or DE in disguise","Recycled algorithms: The open secret of bioinspired optimization","Most bioinspired optimizers are old wine in new bottles","Metaphor-only novelty plagues evolutionary computation"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the criticized algorithms (Harmony Search, Black Hole Optimization, Grey Wolf Optimizer, and similar cases) are typical of the field rather than hand-picked weak examples, so that the paradox of success is a systemic property and not just a curated list.","fun_headline_variants_meta":{"raw":{"variants":["Many 'novel' optimizers are just PSO or DE in disguise","Recycled algorithms: The open secret of bioinspired optimization","Most bioinspired optimizers are old wine in new bottles","Metaphor-only novelty plagues evolutionary computation"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00073,"raw_usage":{"total_tokens":3306,"prompt_tokens":1019,"completion_tokens":2287,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":635,"completion_tokens_details":{"reasoning_tokens":2215}},"tokens_in":635,"tokens_out":2287,"duration_ms":15423,"temperature":1.0,"reasoning_tokens":2215,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T20:38:45.590255+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A random or systematic audit of a sizable sample of published bioinspired algorithms—formalizing each solver's search operators and testing equation-level and configuration-level equivalence against modern PSO, DE, and GA—that found the majority are not equivalent to those three classical families would refute the field-level claim, though individual case studies like Harmony Search or Black Hole Optimization could still stand.","supporting_citations":[{"cited_title":"Molina, J","cited_arxiv_id":null,"evidence_quote":"Supplies the paper's quantitative core: more than half of reviewed nature-inspired solvers are minor variants of PSO, DE, and GA."},{"cited_title":"Campelo, C","cited_arxiv_id":null,"evidence_quote":"Documents the tide of metaphor-based method proposals and the publish-or-perish incentives behind them."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Argues automated design can replace manual metaphor-driven design, grounding the paper's forward-looking pathway."},{"cited_title":"Weyland, A Rigorous Analysis of the Harmony Search Algorithm: How the Research Community can be Misled by a “Novel” Methodology, Int","cited_arxiv_id":null,"evidence_quote":"Early case study showing Harmony Search is essentially a special case of Evolution Strategies, a canonical weak-proposal example."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Shows Black Hole Optimization is a simplification of Particle Swarm Optimization, another canonical weak-proposal example."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides component-level evidence that Grey Wolf, Firefly, and Bat algorithms are reformulations of PSO variants."}],"review_version":1}