Adaptive Hybrid Particle Swarm Optimization with Gradient Descent

Aryan Gurudeo

REVIEW 4 major objections 4 minor 25 references

Adaptive Hybrid Particle Swarm Optimization with Gradient Descent

T0 review · 4 major / 4 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Gradient boost helps swarms only after they find a smooth basin.

desk verdict A genuinely honest hybrid-optimizer study with a rare budget-normalized control, but the control's inertia schedule is under-specified and could flip the positive claims. read the letter →

arxiv 2608.11258 v1 pith:B7XCWQLG submitted 2026-08-07 cs.AI cs.NE

Aryan Gurudeo This is my paper · ORCID

classification cs.AIcs.NE

keywords particleswarmoptimizationgradientdescenthybridadaptiveweightingdiversitysigmoidgatingbudget-normalizedcomparisonconditionalvalueofgradients

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish a conditional claim: fusing gradient descent into particle swarm optimization (PSO) helps only when the swarm has already settled into a basin with smooth local structure, and can be wasted or harmful when applied too early or on rugged landscapes. To make the timing automatic, it introduces Adaptive Hybrid PSO (AHPSO), which scales the gradient step by a sigmoid function of swarm diversity, so gradient influence stays near zero during exploration and rises toward its maximum during exploitation, with no manual phase switch. The paper's budget-normalized experiments show that vanilla PSO with the same total function evaluations wins 52.5% of the tested configurations, while AHPSO wins 20%; the hybrid's genuine advantage is confined to functions whose smooth local basins reward directed descent (F8, F24–F27). A careful reader should care because the paper is attempting to say when -- and when not -- the extra gradient cost is worth paying, instead of claiming a universal improvement.

What carries the argument

The central object is the sigmoid diversity gate, which computes a gradient weight from the swarm's spread: $$\$\alpha$(t) = \alpha_{\min} + \frac{1-\alpha_{\min}}{1+$e^{{k(D(t)/D(0)-\tau)}}$}$$ where $D(t)$ is the average per-dimension population standard deviation of particle positions, $\alpha_{\min}=0.1$, $\tau=0.3$, and $k=5$. After each standard PSO update, every particle takes an extra step of size $\alpha(t)$ times a first-order gradient step, with gradients estimated by central differences at a cost of $2d$ extra function evaluations per particle per iteration. The gate carries the argument because it is the self-correcting timing mechanism: when diversity is high the gradient weight is tiny, when the swarm clusters the weight rises, and if diversity rebounds the weight automatically drops again. The paper uses this gate to test whether the expensive gradient evaluations are spent only where gradients are informative.

What would settle it

Re-run the budget-normalized comparison with two explicit ways of extending vanilla PSO's horizon: (a) inertia decreasing linearly across all 30,500 iterations, and (b) inertia cycling 0.9 to 0.4 every 500 iterations. If PSO-BN's win count over AHPSO changes materially between the two schedules, then the conditional-value conclusion is driven by the control construction rather than by gradient information.

Watch

Extended reading notes

Core claim

The central claim is that gradient injection is a conditional, not universal, improvement over PSO. Under budget normalization, vanilla PSO given equivalent total function evaluations wins 21 of 40 configurations (52.5%), AHPSO wins 8 (20%), and 11 tie, with a rank-based significance test favoring PSO (p = 7.0e-5); AHPSO keeps a strong edge only on functions with smooth local basins (F8, F24–F27), where one directed gradient step beats further undirected sampling even at equal cost. Under iteration-matched comparison, the AHPSO-Adadelta variant ranks first among nine methods, including CMA-ES, but the paper reads that result as evidence of complementary strengths rather than general superiority. The contribution is a characterization of when gradient direction converts function evaluations into directed information: valuable on smooth basins the swarm has found, wasteful otherwise.

Load-bearing premise

The budget-normalized negative result rests on the assumption that running vanilla PSO for 30,500 iterations (instead of 500) to match AHPSO's total function evaluations does not itself change PSO's exploration-exploitation balance; the paper does not specify how the inertia schedule is stretched or repeated over the longer runs, so the conclusion that gradient direction is only conditionally valuable could be an artifact of the control.

Editorial extensions

If this is right

If the conditional claim is right, pairing PSO with a self-tuning first-order optimizer such as Adadelta is the safe default: on most problems it is neutral, on smooth-basin problems it helps, and only a fixed-step optimizer like SGD routinely hurts by diverging.
On smooth unimodal functions, extra iterations alone let vanilla PSO reach machine precision, so the 61x evaluation overhead of finite-difference gradients is pure waste; AHPSO's value appears only when PSO cannot reach the target within its original budget or when final accuracy matters more than time-to-threshold.
On composite, noisy, and other smooth-local-basin functions (F24–F29, F8), a directed gradient step inside an identified basin outperforms additional random sampling at equal cost, meaning evaluation budget is not the only resource that matters -- directional information is a resource too.
AHPSO-Adadelta and CMA-ES score nearly identically overall (20 wins, 20 losses, 2 ties) yet dominate on different function types, which implies explicit first-order gradient information and implicit second-order covariance information exploit different landscape properties and are complementary rather than redundant.
The practical niche is medium-dimensional problems (roughly d = 10 to 30) with cheap function evaluations; at low dimensions CMA-ES is stronger, and with expensive simulations the gradient overhead becomes prohibitive.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

An implication the author leaves implicit is that the budget-normalized result generalizes into a design principle: measure an optimizer's value per unit of directional information, not per function evaluation, because extra undirected evaluations are nearly free on smooth landscapes but useless on rugged ones.
A testable extension, which the paper itself mentions as future work, is replacing finite-difference gradients with automatic differentiation when source code is available; if the 61x overhead disappears, the conditional advantage on smooth-basin functions should widen and may extend to d > 100.
Another extension is to let each particle pick its own optimizer with a multi-armed bandit over recent improvement; a fair test would ask whether per-particle selection reproduces the sigmoid gate's behavior without any explicit diversity signal.
The four-function ablation hints that the sigmoid's benefit is robustness rather than peak performance, since fixed alpha values tuned per function sometimes beat it; a stronger follow-up would compare adaptive gating against per-function-tuned fixed alpha over the entire 42-configuration suite.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Colleague,

You should know two things about this paper. First, it is one of the more honest hybrid-optimizer papers I've seen: the authors explicitly say they are not claiming universal superiority, give PSO an equal evaluation budget, and report the negative result (PSO wins 52.5% of configurations). Second, the budget-normalized control that supports the conditional positive claim has a gap that needs fixing before I'd trust that half.

The new thing here is the sigmoid diversity gate that modulates gradient injection, a neat variation on Zhan's APSO and memetic hybrids. The paper does a lot right: 14,700 runs, comparisons against CMA-ES and CLPSO, a sensitivity analysis, an ablation, two engineering problems, and wall-clock timing. The central finding—that under equal evaluation budgets vanilla PSO beats the hybrid on most problems, with the hybrid retaining an edge only on F8 and F24–F27—is a genuinely useful empirical boundary.

The soft spots are real but not fatal. The main one: the PSO-BN control is under-specified. The paper says it runs 30,500 iterations with "all other parameters remaining identical," but never says how the linear inertia schedule (w from 0.9 to 0.4, normally over 500 iterations) is extended. If it's linearly stretched, then at iteration 500—where AHPSO finishes its PSO phase and starts gradient refinement—PSO-BN still has w≈0.89 and is exploring. You'd be comparing a converged-then-refine method against an explore-long method, which would make the F8/F24–F27 wins an artifact of the control rather than evidence for gradient direction. The authors need to specify the schedule and test alternatives. Second, τ, k, α_min, and the dual learning rates are tuned on the same benchmark suite; the sensitivity analysis helps, but the rankings are still in-sample. Third, Table V says 40 configurations while the design says 42—the paper doesn't reconcile that. Fourth, SHADE/L-SHADE are named as important baselines but not included; that's a gap, though CMA-ES is a solid stand-in.

Who is this for? Practitioners with cheap, smooth, medium-dimensional objectives who want to know whether a gradient-PSO hybrid is worth it. It's a useful empirical datapoint, not a breakthrough. It deserves a serious referee, and I'd accept it for review with the expectation that the control specification gets fixed. If the authors can show the smooth-basin advantage survives a properly specified control, the paper becomes a solid contribution.

Referee Report

4 major / 4 minor

Summary. The paper proposes Adaptive Hybrid PSO (AHPSO), which computes a gradient step for each particle and scales it by a sigmoid function of the swarm's diversity, so that gradient influence is low during exploration and high during exploitation. The authors evaluate six gradient-optimizer variants against vanilla PSO and two baselines (CLPSO, CMA-ES) on 29 benchmark functions with 42 configurations, plus two engineering design problems. They report that under an iteration-matched budget AHPSO-Adadelta ranks first among nine methods, while under a budget-normalized comparison where vanilla PSO receives the same total number of function evaluations, PSO wins 52.5% of configurations and AHPSO wins only 20%. The paper's central claim is that gradient injection is beneficial only when the swarm has found a basin with smooth local structure, and that AHPSO's remaining wins on F8 and F24-F27 constitute evidence that directed descent outperforms undirected sampling when gradients are informative.

Significance. If established, the conditional characterization would be a useful practical guideline and the diversity-gating principle is a clean idea. The paper is unusually honest about its own limitations, acknowledging the 61x overhead, the in-sample tuning of tau, and the oracle-like dual learning rate, and the budget-normalized comparison is a methodologically sound instinct. The statistical testing is thorough, including Friedman tests, Holm-Bonferroni corrections, and effect sizes. However, the central evidence is currently undermined by the unspecified construction of the PSO-BN control and by the lack of an independent validation protocol; the 'smooth local basins' explanation is also assigned post hoc without a measurable definition. The contribution is valuable in potential but not yet established.

major comments (4)

[IV-G, Table V] The budget-normalized control PSO-BN is underspecified: the manuscript states only that 'all other parameters remain identical (N=30, w∈[0.4,0.9], c1=c2=2.0)' without explaining how the linear inertia decrease from 0.9 to 0.4 is applied over 30,500 iterations at d=30 and 10,500 iterations at d=10. If the Shi-Eberhart schedule is stretched linearly, then at the iteration where AHPSO's own PSO phase has reached w=0.4 (iteration 500), PSO-BN would still have w≈0.892 and would remain exploration-dominated for most of its budget. The comparison would then not isolate gradient direction from additional random sampling; it would compare a converged-then-refine algorithm against an explore-long-then-converge-late algorithm. Because the paper's central claim, that AHPSO's wins on F8 and F24-F27 demonstrate a benefit of directed descent that random sampling cannot replicate, rests entirely on this control, the authors should specify the inertia schedule explicitly and, ideally, add a control whose inertia profile matches AHPSO's temporal profile (e.g., the same 500-iteration decrease followed by w=0.4 for the remaining evaluations).
[II-B, III-B, V-Limitations 2 and 4] The hyperparameters τ, k, α_min, and the dual learning rate η are selected on the same benchmark suite used for evaluation. Section V limitation 2 admits that τ was tuned on this suite, and limitation 4 admits that η=0.01/0.001 requires problem-class knowledge and that the authors deliberately reported best-case settings for each optimizer. The Friedman rankings in Tables II and IV are therefore in-sample estimates, and the reported p-values do not account for the tuning selection. To support the claim that AHPSO is a principled mechanism rather than a per-benchmark fit, the authors should add an independent validation set, use a nested tuning/evaluation protocol, or report results under a single universal hyperparameter set (for example, fixing η=0.01 for all functions or relying only on optimizers that ignore η).
[IV-J, Table IX, Fig. 9 caption] The ablation interpretation is not supported by the reported numbers. In Table IX, the sigmoid achieves the best median on F1 and F12, but on F9 its median of 6.80 is the worst of the six entries (the fixed α=0.5 median is 5.06), and on F11 its median of 0.105 is second-worst (α=1.0 reaches 0.081). The text claims the sigmoid provides 'best or near-best performance across all function types' and that its value is robustness, but the table shows the sigmoid is worst on one function type and second-worst on another. In addition, the caption of Fig. 9 states that composite functions 'show mostly ties because rotation destroys axis-aligned gradient structure,' which is inconsistent with Table V listing AHPSO wins on F24-F27, a subset of composite functions; this contradiction should be resolved.
[IV-G Interpretation and V Conclusion] The central explanation that AHPSO wins on F8 and F24-F27 because these functions have 'smooth local basins' is assigned post hoc and is not operationalized. The paper provides no quantitative landscape metric (local curvature, basin size, finite-difference accuracy, or gradient informativeness) that distinguishes F8 and F24-F27 from F1-F4, F9, or F12-F13, where PSO-BN wins. Without a measurable definition of basin smoothness, the conditional claim is a restatement of the outcome rather than a characterization. A synthetic experiment that varies basin smoothness and exhibits the predicted crossover, or a concrete measure computed on the benchmark functions, would substantiate the paper's main conclusion.

minor comments (4)

[III-B, Fig. 8] Section III.B states that all experiments use 50 independent runs, but the caption of Fig. 8 reports 'median of 15 runs'; please state which number applies to the convergence curves and reconcile the discrepancy.
[Table V] Table V reports outcomes as wins/losses/ties without defining the criterion; specify whether these counts are based on medians, means, or statistically significant differences, and whether any multiple-testing correction is applied.
[IV-F] In Section IV.F, the pairwise comparison between CMA-ES and AHPSO-Adadelta (20 wins, 20 losses, 2 ties) is presented without stating whether the counts are based on raw medians or on the Holm-Bonferroni-corrected Mann-Whitney tests used in Table III; clarify the protocol.
[Table I] The paper would benefit from a table that maps the benchmark function numbers (F1-F29) to their standard mathematical definitions and to the claimed landscape categories, so that the 'smooth local basin' classification can be checked.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; the central claims are empirical, tuning is disclosed, and no equation-level reduction to inputs is present.

full rationale

The paper's main claims are empirical rather than derived, and no load-bearing step reduces to its own inputs by construction. The adaptive sigmoid in Eq. (5) is a design choice with explicitly stated hyperparameters, and the update in Eq. (6) simply applies a gradient step scaled by alpha; the conclusion that gradient injection helps only on smooth-basin problems is drawn from benchmark comparisons, not from the equations themselves. The closest candidate for circularity is the in-sample tuning of tau=0.3 and k=5 on the same benchmark suite, which is acknowledged in Limitation 2, along with the dual learning rates acknowledged in Limitation 4. This is a disclosed validity concern rather than a circular reduction, and the sensitivity analysis in Tables VI and VII shows the rankings are stable over tau in [0.2,0.4] and k in [3,10], so the central conditional claim is not statistically forced by these fitted values. The budget-normalized control PSO-BN leaves the inertia schedule unspecified over its 30,500-iteration horizon, which is a legitimate control-validity criticism, but it is an omission in the experimental design, not an equation-level equivalence between the control and the claimed result. No self-citation is load-bearing, no uniqueness theorem is imported from the authors, and no known result is merely renamed with new coordinates. The external engineering problems in Section IV-I provide an independent, if limited, check. Accordingly, no circular step is identified.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The paper is an empirical study; its central claims depend on a handful of manually chosen or tuned hyperparameters (τ, k, α_min, η, ϵ) and on assumptions about diversity as a state signal, finite-difference gradient quality, benchmark representativeness, and fairness of the budget-normalized control. No new physical or algorithmic entities are postulated.

free parameters (5)

τ (sigmoid midpoint) = 0.3
Tuned on the benchmark suite (Section II-B; limitation 2). Sensitivity sweep shows stability in [0.2, 0.4], but the default is selected on the same test problems used for evaluation.
k (sigmoid steepness) = 5
Set by the author; sensitivity sweep over {3,5,10} shows softer transitions slightly better, so the value is not uniquely determined by data.
α_min (minimum gradient weight) = 0.1
Chosen by hand so gradients never fully dominate; no independent justification is provided.
η (base learning rate) = 0.01 (unimodal), 0.001 (multimodal)
Requires a priori problem-class knowledge, acknowledged as oracle information. Affects SGD, Adagrad, and RMSprop results; Adadelta ignores it entirely.
ϵ (finite-difference step) = 1e-8
Standard central-difference perturbation, chosen by hand. It affects gradient estimates on ill-conditioned or noisy functions.

assumptions (5)

domain assumption Diversity ratio D(t)/D(0) captures the exploration-exploitation state of the swarm (Eqs. 4-5).
The entire gating mechanism relies on this; the paper gives intuitive justification but no independent empirical validation that diversity is a sufficient signal across function types.
domain assumption Central finite differences with ϵ=1e-8 provide usable gradient estimates on all tested continuous benchmark functions (Eq. 7).
On noisy, discontinuous, or ill-conditioned functions the estimates may be misleading; the paper explicitly limits the method to continuous landscapes.
domain assumption The benchmark suite of 29 functions from [2] is representative of real optimization problems relevant to the claim.
Conclusions about when gradients help are drawn from this suite; no external problem class is used to validate the 'smooth local basins' characterization.
domain assumption PSO-BN with 30,500 iterations receives an equivalent but fair budget when all other parameters are 'identical' (Section IV-G).
The extended inertia schedule and long-run dynamics are not specified; the fairness of the budget-normalized comparison depends on this assumption.
standard math No-Free-Lunch theorem [25] justifies that no single algorithm dominates, framing the conditional result.
Used as background framing only; not quantified or derived in this paper.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Adaptive Hybrid Particle Swarm Optimization with Gradient Descent." pith.science (2026). https://pith.science/paper/B7XCWQLG

@misc{pith2026260811258,
  author       = {Pith},
  title        = {Pith review of: Adaptive Hybrid Particle Swarm Optimization with Gradient Descent},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/B7XCWQLG}},
  note         = {Machine review of arXiv:2608.11258}
}

read the original abstract

Gradient injection helps Particle Swarm Optimization (PSO) only when the swarm has identified a basin with smooth local structure, not universally. We propose Adaptive Hybrid PSO (AHPSO), which uses a sigmoid function on swarm diversity to automatically modulate gradient influence: near-zero during exploration, near-maximum during exploitation, with no manual phase-switching. Under budget-normalized comparison (PSO given equivalent total function evaluations), PSO wins 52.5% of 40 configurations versus AHPSO's 20% (p = 7.0e-5, Friedman). AHPSO retains advantage specifically on problems with smooth local basins (F8, F24-F27) where directed descent outperforms undirected sampling even at equal cost. Under iteration-matched comparison across 29 functions (42 configurations, 14,700 runs), AHPSO-Adadelta ranks first of 9 methods including CMA-ES (p = 9.75e-4). The contribution is a principled characterization of when gradient injection provides value in swarm-based search, not a claim of universal superiority.

Figures

Figures reproduced from arXiv: 2608.11258 by the authors.

**Figure 2.** PSO velocity update. Each particle is pulled by three forces: inertia [PITH_FULL_IMAGE:figures/full_fig_p001_2.png] view at source ↗

**Figure 3.** The sigmoid function that controls gradient influence. When particles [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗

Figures from the paper (6 more)

**Figure 4.** Figure 4: A typical run: diversity (blue) decays as particles converge, causing the [PITH_FULL_IMAGE:figures/full_fig_p003_4.png]

**Figure 5.** Figure 5: AHPSO pipeline showing the feedback loop that makes the method adaptive. Steps (1), (2) are standard PSO; the novelty is steps (3), (4): diversity [PITH_FULL_IMAGE:figures/full_fig_p010_5.png]

**Figure 6.** Figure 6: Performance scores for each method, computed as [PITH_FULL_IMAGE:figures/full_fig_p011_6.png]

**Figure 7.** Figure 7: Wins and losses by function group. Unimodal functions show the [PITH_FULL_IMAGE:figures/full_fig_p011_7.png]

**Figure 9.** Figure 9: Statistical significance of each hybrid vs. standard PSO across all 42 [PITH_FULL_IMAGE:figures/full_fig_p011_9.png]

**Figure 8.** Figure 8: Convergence curves (median of 15 runs, IQR shading shows 25th– [PITH_FULL_IMAGE:figures/full_fig_p011_8.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

25 extracted references · 21 canonical work pages

[1]

Particle swarm optimization,

J. Kennedy and R. Eberhart, “Particle swarm optimization,” inProc. IEEE ICNN, 1995, pp. 1942–1948

work page 1995
[2]

Grey wolf optimizer,

S. Mirjalili, S. M. Mirjalili, and A. Lewis, “Grey wolf optimizer,” Advances in Engineering Software, vol. 69, pp. 46–61, 2014

2014
[3]

Adam: A method for stochastic optimization,

D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” inProc. ICLR, 2015

2015
[4]

ADADELTA: An adaptive learning rate method,

M. D. Zeiler, “ADADELTA: An adaptive learning rate method,” arXiv:1212.5701, 2012

arXiv 2012
[5]

Adaptive subgradient methods for online learning and stochastic optimization,

J. Duchi, E. Hazan, and Y . Singer, “Adaptive subgradient methods for online learning and stochastic optimization,”JMLR, vol. 12, pp. 2121– 2159, 2011

work page 2011
[6]

RMSProp: Divide the gradient by a running average of its recent magnitude,

T. Tieleman and G. Hinton, “RMSProp: Divide the gradient by a running average of its recent magnitude,”COURSERA: Neural Networks for Machine Learning, Lecture 6.5, 2012

work page 2012
[7]

Incorporating Nesterov momentum into Adam,

T. Dozat, “Incorporating Nesterov momentum into Adam,” inICLR Workshop, 2016

work page 2016
[8]

Simulation of a new hybrid particle swarm optimization algorithm,

M. M. Noel and T. C. Jannett, “Simulation of a new hybrid particle swarm optimization algorithm,” inProc. 36th Southeastern Symp. System Theory, 2004, pp. 150–153

work page 2004

Show all 25 references

[9]

A modified particle swarm optimizer,

Y . Shi and R. Eberhart, “A modified particle swarm optimizer,” inProc. IEEE Int. Conf. Evol. Comput., 1998, pp. 69–73

1998
[10]

Adaptive particle swarm optimization,

Z.-H. Zhan, J. Zhang, Y . Li, and H. S.-H. Chung, “Adaptive particle swarm optimization,”IEEE Trans. Syst., Man, Cybern. B, vol. 39, no. 6, pp. 1362–1381, 2009

2009
[11]

Comprehensive learning particle swarm optimizer for global optimization of multimodal functions,

J. J. Liang, A. K. Qin, P. N. Suganthan, and S. Baskar, “Comprehensive learning particle swarm optimizer for global optimization of multimodal functions,”IEEE Trans. Evol. Comput., vol. 10, no. 3, pp. 281–295, 2006

2006
[12]

The CMA evolution strategy: A comparing review,

N. Hansen, “The CMA evolution strategy: A comparing review,” in Towards a New Evolutionary Computation, J. A. Lozano et al., Eds. Berlin: Springer, 2006, pp. 75–102

2006
[13]

Improving the search performance of SHADE using linear population size reduction,

R. Tanabe and A. S. Fukunaga, “Improving the search performance of SHADE using linear population size reduction,” inProc. IEEE CEC, 2014, pp. 1658–1665

2014
[14]

Meta-Lamarckian learning in memetic algorithms,

Y . S. Ong and A. J. Keane, “Meta-Lamarckian learning in memetic algorithms,”IEEE Trans. Evol. Comput., vol. 8, no. 2, pp. 99–110, 2004

2004
[15]

Self-organizing hierarchical particle swarm optimizer with time-varying acceleration coefficients,

A. Ratnaweera, S. K. Halgamuge, and H. C. Watson, “Self-organizing hierarchical particle swarm optimizer with time-varying acceleration coefficients,”IEEE Trans. Evol. Comput., vol. 8, no. 3, pp. 240–255, 2004

2004
[16]

A hybrid particle swarm optimization with local search strategy,

S. K. S. Fan and E. Yan, “A hybrid particle swarm optimization with local search strategy,”Applied Soft Computing, vol. 27, pp. 459–472, 2015

2015
[17]

A simple sequentially rejective multiple test procedure,

S. Holm, “A simple sequentially rejective multiple test procedure,” Scandinavian Journal of Statistics, vol. 6, no. 2, pp. 65–70, 1979

1979
[18]

The use of ranks to avoid the assumption of normality implicit in the analysis of variance,

M. Friedman, “The use of ranks to avoid the assumption of normality implicit in the analysis of variance,”J. Amer. Statist. Assoc., vol. 32, no. 200, pp. 675–701, 1937

1937
[19]

Use of a self-adaptive penalty approach for engineering optimization problems,

C. A. Coello Coello, “Use of a self-adaptive penalty approach for engineering optimization problems,”Computers in Industry, vol. 41, no. 2, pp. 113–127, 2000

2000
[20]

A practical tutorial on the use of nonparametric statistical tests as a methodology for comparing evolutionary and swarm intelligence algorithms,

J. Derrac, S. Garc ´ıa, D. Molina, and F. Herrera, “A practical tutorial on the use of nonparametric statistical tests as a methodology for comparing evolutionary and swarm intelligence algorithms,”Swarm and Evolutionary Computation, vol. 1, no. 1, pp. 3–18, 2011

2011
[21]

Particle swarm optimization for single objective continuous space problems: A review,

M. R. Bonyadi and Z. Michalewicz, “Particle swarm optimization for single objective continuous space problems: A review,”Evolutionary Computation, vol. 25, no. 1, pp. 1–54, 2017

2017
[22]

Evolving cog- nitive and social experience in particle swarm optimization through dif- ferential evolution: A hybrid approach,

M. G. Epitropakis, V . P. Plagianakos, and M. N. Vrahatis, “Evolving cog- nitive and social experience in particle swarm optimization through dif- ferential evolution: A hybrid approach,”Information Sciences, vol. 216, pp. 50–92, 2012

2012
[23]

Combining gradient techniques for numerical multi-objective evolutionary optimization,

P. A. N. Bosman and E. D. de Jong, “Combining gradient techniques for numerical multi-objective evolutionary optimization,” inProc. GECCO, 2005, pp. 627–634

2005
[24]

Two-layer particle swarm optimization with intelligent division of labor,

W. H. Lim and N. A. M. Isa, “Two-layer particle swarm optimization with intelligent division of labor,”Engineering Applications of Artificial Intelligence, vol. 26, no. 10, pp. 2263–2279, 2014

2014
[25]

No free lunch theorems for optimization,

D. H. Wolpert and W. G. Macready, “No free lunch theorems for optimization,”IEEE Trans. Evol. Comput., vol. 1, no. 1, pp. 67–82, 1997. 10 PSO Update Measure Diversity Compute (t) GD Step × New xi Eqs. 1-2 Eq. 3 Eq. 4 (sigmoid) Eq. 5 High diversity small (explore) | Low diversi...

1997

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.

[1] [1]

Particle swarm optimization,

J. Kennedy and R. Eberhart, “Particle swarm optimization,” inProc. IEEE ICNN, 1995, pp. 1942–1948

work page 1995

[2] [2]

Grey wolf optimizer,

S. Mirjalili, S. M. Mirjalili, and A. Lewis, “Grey wolf optimizer,” Advances in Engineering Software, vol. 69, pp. 46–61, 2014

2014

[3] [3]

Adam: A method for stochastic optimization,

D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” inProc. ICLR, 2015

2015

[4] [4]

ADADELTA: An adaptive learning rate method,

M. D. Zeiler, “ADADELTA: An adaptive learning rate method,” arXiv:1212.5701, 2012

arXiv 2012

[5] [5]

Adaptive subgradient methods for online learning and stochastic optimization,

J. Duchi, E. Hazan, and Y . Singer, “Adaptive subgradient methods for online learning and stochastic optimization,”JMLR, vol. 12, pp. 2121– 2159, 2011

work page 2011

[6] [6]

RMSProp: Divide the gradient by a running average of its recent magnitude,

T. Tieleman and G. Hinton, “RMSProp: Divide the gradient by a running average of its recent magnitude,”COURSERA: Neural Networks for Machine Learning, Lecture 6.5, 2012

work page 2012

[7] [7]

Incorporating Nesterov momentum into Adam,

T. Dozat, “Incorporating Nesterov momentum into Adam,” inICLR Workshop, 2016

work page 2016

[8] [8]

Simulation of a new hybrid particle swarm optimization algorithm,

M. M. Noel and T. C. Jannett, “Simulation of a new hybrid particle swarm optimization algorithm,” inProc. 36th Southeastern Symp. System Theory, 2004, pp. 150–153

work page 2004