REVIEW 3 major objections 4 minor 41 references
Efficient Fireworks Algorithm Equipped with an Explosion Mechanism based on Student's T-distribution
T0 review · 3 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read A fireworks algorithm that samples from a Student's t-distribution, with updates derived from natural gradient descent, claims to match or beat state-of-the-art optimizers on standard benchmarks.
desk verdict A credible empirical FWA variant with an over-sold natural-gradient story; the benchmarks deserve a referee, but the 'solid theoretical foundation' claim needs to be walked back or quantified. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the Fisher information matrix of the multivariate Student's t-distribution, equation (17), and its inverse. To make the inverse tractable the paper drops the trace-trace cross term, leaving the per-block inverses $F_m^{-1}=\frac{d+v+2}{d+v}\Sigma$ and the corresponding scale update (47); these, fused with CMA-ES-style rank weights, define the explosion weights of Algorithm 1. A second mechanism, AdjustDegreeOfFreedom, increases the degrees of freedom when a better solution is found, letting the sampler interpolate between heavy-tailed exploration and Gaussian-like exploitation, while a loser-out tournament restarts unpromising fireworks.
What would settle it
Run TFWA on a rotated ill-conditioned quadratic such as the CEC elliptic function and record, at each generation, the magnitude of the dropped term $-\frac{1}{2(d+v+2)}\operatorname{tr}(\partial\Sigma/\partial\theta_i\,\Sigma^{-1})\operatorname{tr}(\partial\Sigma/\partial\theta_j\,\Sigma^{-1})$ in equation (40) relative to the kept curvature term. If it is not at least an order of magnitude smaller, the inverse-Fisher update is not the correct natural gradient, contradicting the paper's stated theoretical foundation.
Extended reading notes
Core claim
The central discovery is that a Student's t-distribution explosion operator can be derived from natural gradient descent, not just grafted on heuristically. Starting from the t-density and its Fisher information matrix, the paper obtains natural-gradient update rules for the mean and the scale matrix, equations (44) and (47), which generalize the familiar Gaussian updates of CMA-ES and reduce to them as the degrees of freedom grow. The algorithm then fuses these natural-gradient weights with rank-based weights, so each sampled spark contributes to reshaping the sampling distribution in proportion to its fitness. The adaptive degrees-of-freedom rule completes the mechanism: when a firework improves, the sampler becomes more Gaussian and exploits; when it stagnates, the heavy tails remain and exploration continues. The paper's claim is that this combination makes TFWA the strongest fireworks-algorithm variant and competitive with state-of-the-art optimizers on CEC2013 and CEC2017, with particular strength on unimodal problems and on functions with many extreme points.
Load-bearing premise
The derivation's validity rests on treating the trace-trace term of the Fisher information matrix as negligible; if the scale matrix becomes ill-conditioned, that term can be large and the resulting update is no longer the true natural gradient.
Editorial extensions
If this is right
- TFWA reaches the global optimum on unimodal and convex problems in the CEC suites, closing the gap that previously kept fireworks-algorithm variants far behind SOTA on such problems.
- The degree-of-freedom adaptation automatically shifts the sampler from heavy-tailed exploration to Gaussian-like exploitation, making the separate guiding-spark mutation operator unnecessary.
- On multimodal CEC2017 functions with many local extrema, TFWA beats Gaussian-sampling algorithms such as RB-IPOP-CMA-ES, indicating that heavy-tailed explosion sampling helps escape many local optima.
- On the CEC2017 benchmark TFWA places second to the competition winner EBOwithCMAR and ahead of two other CMA-based SOTA variants, while on CEC2013 30-D it achieves the best average rank among all compared algorithms.
Reading between the lines
- Because the trace-trace approximation has no error bound, TFWA's claimed 'solid theoretical foundation' likely holds in the regime where the scale matrix is near-isotropic or the degrees of freedom are large; on strongly rotated, ill-conditioned landscapes, the empirical gains may come as much from the restart and degree-of-freedom schedules as from the natural-gradient correction. This is an edit
- The same derivation could be applied to other heavy-tailed families or to other swarm optimizers that currently sample with Gaussians, giving each a tunable exploration tail without changing the underlying selection skeleton.
- A direct ablation—running TFWA with the full Fisher inverse instead of the approximated one on rotated CEC functions—would settle whether the approximation is actually harmless in practice.
- The trajectory of the degrees of freedom across a run could serve as a cheap diagnostic of landscape difficulty: functions where the degrees of freedom grow slowly are the ones where the optimizer is still exploring.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes TFWA, a fireworks algorithm variant whose explosion sampling uses a multivariate Student's t-distribution. The distribution parameters are updated using a natural-gradient-descent-style derivation for the t-distribution, with a degree-of-freedom adaptation rule and a loser-out restart mechanism adopted from LoTFWA. The algorithm is evaluated on CEC2013 (30 and 50 dimensions) and CEC2017 (30 dimensions) with a budget of 10,000d evaluations and 30 independent runs, comparing against two FWA variants and several state-of-the-art competitors. The central claims are that TFWA is the strongest FWA variant, that it matches or surpasses top CEC2013/CEC2017 algorithms on average rank, and that the t-distribution sampling gives a theoretically founded generalization of Gaussian-based CMA-ES sampling.
Significance. If the claims hold, TFWA would be a notable contribution: it extends the information-geometric derivation of CMA-ES to a tunable heavy-tailed family, demonstrates that a t-distribution sampler can beat Gaussian-based CMA-ES variants on multimodal problems, and strengthens the FWA line of research. The experimental study is substantial: standard evaluation budget, 30 independent runs, one-to-one Wilcoxon tests, and per-function tables plus convergence plots. The derivation is presented step-by-step in the appendix, which is valuable even where it needs repair. The paper's main weakness is that the load-bearing theoretical approximation (dropping the trace-trace Fisher term) is not justified with any error bound, and the pseudo-code has notational/indexing errors that prevent exact reproduction. With those fixed, the contribution would be appropriate for the journal.
major comments (3)
- [Appendix, Eq. (40) and (42)-(47)] The derivation of the inverse Fisher information for the scale matrix relies on setting the trace-trace term -1/(2(d+v+2)) tr(∂Σ/∂θ_i Σ^{-1}) tr(∂Σ/∂θ_j Σ^{-1}) to zero. The argument given after Eq. (40) (bounded two-norm of the transformation matrix, d+v+2 ≥ v, and large empirical v) does not bound the size of this term relative to the kept term. On rotated ill-conditioned CEC functions, Σ can have condition number 10^6 or larger, and the trace-trace term along directions associated with small eigenvalues can be comparable to or larger than the kept term. Because Eqs. (42), (44), (45), and (47) all inherit this approximation, the claimed natural-gradient update for the t-distribution is not established. Please provide an explicit error bound, or a numerical test measuring the Frobenius norm of the dropped term relative to the retained block along a representative TFWA trajectory on, for example, CEC2013 f15/f16 or CEC2017 f10.
- [Algorithm 1, lines 8, 11-13] The pseudo-code as written is not executable and the update is not translation invariant. Line 8 computes w'_i using pop[i]^T C^{-1} pop[i] without centering on the current mean m; the theory in Eq. (44) uses (x-m). Line 11 m' ← w'' * pop is a weighted mean only if pop is sorted and the product is a sum over the λ samples, which is not stated; as written it is a matrix expression. Line 12 sets cov_i to the scalar (pop_i - m_i)^T (pop_i - m_i), whereas the covariance update requires the outer product (pop_i - m_i)(pop_i - m_i)^T. In addition, the rank-based weights w_i in line 2 are meaningful only after sorting the sampled population by fitness, but no fitness-sorting step appears before line 9. These issues make the algorithm impossible to reproduce exactly and must be corrected with a precise, self-contained pseudocode.
- [Tables VI-VIII and Section IV.B] The comparison against NBIPOPaCMA, SHADE, MVMO, SPSO2011, EBOwithCMAR, LSHADE-SPACMA, and RB-IPOP-CMA-ES is presented as evidence that TFWA 'matches or surpasses SOTA' and is 'far superior' on some functions. The paper does not state whether the baseline results were re-run under the same conditions (budget, initialization, bound handling, number of runs) or taken from published competition reports. If the latter, the comparison is not controlled: CEC competition results often use different population sizes, restarts, and evaluation budgets. Please specify the provenance of every baseline column and, ideally, provide local re-runs with identical experimental protocol, or explicitly restrict the claim to a comparison against previously reported numbers.
minor comments (4)
- [Section II.B, Eqs. (3)-(4)] The tail bounds are derived only for a ≥ 0; please state that the argument is symmetric and valid for |a| or specify the one-sided tail being bounded.
- [Appendix, paragraph after Eq. (40)] The sentence 'F is a diagonal matrix' is inaccurate; the Fisher information is block-structured, and the covariance block is not diagonal in general. Please rephrase as 'block-diagonal between the mean and scale parameters' or qualify the statement.
- [Algorithm 2, step 2] The update df' ← max(df*factor, df+1) is unusual because for small df the df+1 branch dominates; please clarify whether the intent is to guarantee a minimum increase of 1 and describe the rationale.
- [Throughout] The paper contains inconsistent spacing in 'FW A' and inconsistent capitalization of 'student's T'; please standardize these terms.
Circularity Check
No significant circularity: the t-distribution natural-gradient derivation is self-contained, and the performance claims are tested against external CEC benchmarks rather than being constructed from fitted values.
full rationale
The paper's central theoretical derivation computes the Fisher information of the multivariate Student's t-distribution from the density in Eq. (5) using explicit integrals in the appendix (Eqs. 20-47). The resulting natural-gradient weights (Eqs. 18-19 and 44, 47) are derived from the distributional model itself, not from benchmark optima or fitted parameters. The dropped trace-trace term in Eq. (40) is an acknowledged approximation described as 'for the convenience of the theoretical derivation'; this is a rigor and correctness concern (no error bound is provided), but it is not a circular step because the approximation does not assume the conclusion. The self-citations present in the paper, such as adopting the LoTFWA restart strategy [11] and using LoTFWA/MGFWA [11,12] as baselines, are algorithmic components and comparative references; they are not used to justify the central theoretical claim, and the benchmarks (CEC2013, CEC2017) are external and independent of the authors' fitted values. The degree-of-freedom adjustment heuristic is presented without an ablation, but it is not called a prediction nor is it derived from the benchmarks. No equation in the paper reduces to a fitted input or to a self-citation chain, so the derivation chain is not circular.
Assumptions & free parameters
free parameters (5)
- number of fireworks N =
2
- initial degrees of freedom df =
5
- degree-of-freedom factor array =
[1.05, 10]
- sparks per generation lambda =
10d/N
- CMA-ES-style learning rates and scale parameters =
Table I formulas
assumptions (7)
- standard math Multivariate Student's t-distribution density and its moments (Eq. 5-7)
- domain assumption Natural gradient descent framework of Amari and its CMA-ES interpretation (Akimoto et al.)
- ad hoc to paper Trace-trace term of the Fisher information for covariance is negligible
- ad hoc to paper Fusing rank-based weights with natural-gradient weights produces a valid update
- ad hoc to paper Degree-of-freedom adaptation rule
- domain assumption Loser-out tournament restart from LoTFWA
- ad hoc to paper Removal of the guiding-spark mutation operator is beneficial
Cite this review
Pith. "Pith review of Efficient Fireworks Algorithm Equipped with an Explosion Mechanism based on Student's T-distribution." pith.science (2026). https://pith.science/paper/NA4ML37X
@misc{pith2026250608484,
author = {Pith},
title = {Pith review of: Efficient Fireworks Algorithm Equipped with an Explosion Mechanism based on Student's T-distribution},
year = {2026},
howpublished = {\url{https://pith.science/paper/NA4ML37X}},
note = {Machine review of arXiv:2506.08484}
}
read the original abstract
Many real-world problems can be transformed into optimization problems, which can be classified into convex and non-convex. Although convex problems are almost completely studied in theory, many related algorithms to many non-convex problems do not work well and we need more optimization techniques. As a swarm intelligence optimization algorithm, the Fireworks Algorithm(FWA) has been widely studied and applied to many real-world scenarios, even including large language model fine-tuning. But the current fireworks algorithm still has a number of problems. Firstly, as a heuristic algorithm, its performance on convex problems cannot match the SOTA results, and can even be said to be unsatisfactory; secondly, the sampling methods (explosion) of most FWA variants are still uniform sampling, which is actually inefficient in high dimensional cases. This work of ours proposes a new student's t-distribution based FWA(TFWA) with a solid theoretical foundation, which fully utilizes the advantage that student's t-distribution can adjust the parameters (degrees of freedom) and thus adjust the exploitation capability. We have fully experimented on mainstream benchmarks CEC2013 and CEC2017, which proves that TFWA not only becomes the strongest variant of the fireworks algorithm, but also achieves results comparable to SOTA on the test set, and its performance is far superior to that of the SOTA algorithm in some scenarios with a large number of extreme points.
Figures
Figures from the paper (7 more)
Reference graph
Works this paper leans on
-
[1]
Gao, J., Li, P., Chen, Z., & Zhang, J. (2020). A survey on deep learning for multimodal data fusion. Neural Computation, 32(5), 829-864
work page 2020
-
[2]
Nguyen, H. H., & Chan, C. W. (2006). Applications of artificial intelligence for optimization of compressor scheduling. Engineering Applications of Artificial Intelligence, 19(2), 113-126
work page 2006
-
[3]
Artificial intel732 ligence applied to stock market trading: a review
-
[4]
Elsken, T., Metzen, J. H., & Hutter, F. (2019). Neural Architecture Search: A Survey, Journal of Machine Learning Research
work page 2019
-
[5]
Holland, J. H. (1992). Genetic algorithms. Scientific american, 267(1), 66-73
work page 1992
-
[6]
Marini, F., & Walczak, B. (2015). Particle swarm optimization (PSO). A tutorial. Chemometrics and Intelligent Laboratory Systems, 149, 153- 165
work page 2015
-
[7]
Dorigo, M., Birattari, M., & Stutzle, T. (2006). Ant colony optimization. IEEE computational intelligence magazine, 1(4), 28-39
work page 2006
-
[8]
Qin, A. K., Huang, V . L., & Suganthan, P. N. (2008). Differential evolution algorithm with strategy adaptation for global numerical op- timization. IEEE transactions on Evolutionary Computation, 13(2), 398- 417
work page 2008
Show all 41 references
-
[9]
Tan, Y ., & Zhu, Y . (2010). Fireworks algorithm for optimization. In Advances in Swarm Intelligence: First International Conference, ICSI 2010, Beijing, China, June 12-15, 2010, Proceedings, Part I 1 (pp. 355- 364). Springer Berlin Heidelberg
2010
-
[10]
(2021, June)
Chen, M., & Tan, Y . (2021, June). Exponentially decaying explosion in fireworks algorithm. In 2021 IEEE Congress on Evolutionary Compu- tation (CEC) (pp. 1406-1413). IEEE
2021
-
[11]
Li, J., & Tan, Y . (2017). Loser-out tournament-based fireworks algorithm for multimodal function optimization. IEEE Transactions on Evolution- ary Computation, 22(5), 679-691
2017
-
[12]
Meng, X., & Tan, Y . (2024). Multi-guiding spark fireworks algorithm: Solving multimodal functions by multiple guiding sparks in fireworks algorithm. Swarm and Evolutionary Computation, 85, 101458
2024
-
[13]
Jin, F., Liu, Y ., & Tan, Y . (2024). Derivative-free optimization for low- 9 TABLE VII: Comparing on CEC2013, dim = 50 idx NBIPOPaCMA.meanNBIPOPaCMA.stdNIPOPaCMA.meanNIPOPaCMA.stdSHADE.mean SHADE.std MVMO.mean MVMO.std SPSO2011.meanSPSO2011.std TFW A.meanTFW A.std 1 0.000e+00...
2024
-
[14]
J., Shen, Y ., Wallis, P., Allen-Zhu, Z., Li, Y ., Wang, S.,
Hu, E. J., Shen, Y ., Wallis, P., Allen-Zhu, Z., Li, Y ., Wang, S., ... & Chen, W. (2021). Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685
2021 arXiv
-
[15]
J., Qu, B
Liang, J. J., Qu, B. Y ., Suganthan, P. N., & Hern ´andez-D´ıaz, A. G. (2013). Problem definitions and evaluation criteria for the CEC 2013 special session on real-parameter optimization. Computational Intelligence Laboratory, Zhengzhou University, Zhengzhou, China and Nanyang...
2013
-
[16]
Wu, G., Mallipeddi, R., & Suganthan, P. N. (2017). Problem definitions and evaluation criteria for the CEC 2017 competition on constrained real-parameter optimization. National University of Defense Technol- ogy, Changsha, Hunan, PR China and Kyungpook National University, Dae...
2017
-
[17]
(2014, July)
Li, J., Zheng, S., & Tan, Y . (2014, July). Adaptive fireworks algorithm. In 2014 IEEE Congress on evolutionary computation (CEC) (pp. 3214- 3221). IEEE
2014
-
[18]
(2014, July)
Zheng, S., Janecek, A., Li, J., & Tan, Y . (2014, July). Dynamic search in fireworks algorithm. In 2014 IEEE Congress on evolutionary computation (CEC) (pp. 3222-3229). IEEE
2014
-
[19]
Li, J., Zheng, S., & Tan, Y . (2016). The effect of information utilization: Introducing a novel guiding spark in the fireworks algorithm. IEEE Transactions on Evolutionary Computation, 21(1), 153-166
2016
-
[20]
Li, Y ., & Tan, Y . (2022). Fireworks algorithm based on search space partition. International Journal of Intelligent Systems, 37(12), 10783- 10812
2022
-
[21]
Chen, M., & Tan, Y . (2023). SF-FW A: A Self-Adaptive Fast Fireworks Algorithm for effective large-scale optimization. Swarm and Evolution- ary Computation, 80, 101314
2023
-
[22]
(1996, May)
Hansen, N., & Ostermeier, A. (1996, May). Adapting arbitrary normal mutation distributions in evolution strategies: The covariance matrix adaptation. In Proceedings of IEEE international conference on evo- lutionary computation (pp. 312-317). IEEE
1996
-
[23]
Akimoto, Y ., Nagata, Y ., Ono, I., & Kobayashi, S. (2012). Theoretical foundation for CMA-ES from information geometry perspective. Algo- rithmica, 64, 698-716
2012
-
[24]
(2013, June)
Loshchilov, I. (2013, June). CMA-ES with restarts for solving CEC 2013 benchmark problems. In 2013 IEEE Congress on Evolutionary Computation (pp. 369-376). Ieee
2013
-
[25]
(2013, June)
Tanabe, R., & Fukunaga, A. (2013, June). Evaluating the performance of SHADE on CEC 2013 benchmark problems. In 2013 IEEE Congress on evolutionary computation (pp. 1952-1959). IEEE
2013
-
[26]
L., & Erlich, I
Rueda, J. L., & Erlich, I. (2013, June). Hybrid mean-variance mapping optimization for solving the IEEE-CEC 2013 competition problems. In 2013 IEEE Congress on Evolutionary Computation (pp. 1664-1671). IEEE
2013
-
[27]
(2013, June)
Zambrano-Bigiarini, M., Clerc, M., & Rojas, R. (2013, June). Standard 10 particle swarm optimisation 2011 at cec-2013: A baseline for future pso improvements. In 2013 IEEE congress on evolutionary computation (pp. 2337-2344). IEEE
2013
-
[28]
K., & Singh, D
Kumar, A., Misra, R. K., & Singh, D. (2017, June). Improving the local search capability of effective butterfly optimizer using covariance matrix adapted retreat phase. In 2017 IEEE congress on evolutionary computation (CEC) (pp. 1835-1842). IEEE
2017
-
[29]
W., Hadi, A
Mohamed, A. W., Hadi, A. A., Fattouh, A. M., & Jambi, K. M. (2017, June). LSHADE with semi-parameter adaptation hybrid with CMA-ES for solving CEC 2017 benchmark problems. In 2017 IEEE Congress on evolutionary computation (CEC) (pp. 145-152). IEEE
2017
-
[30]
(2017, June)
Biedrzycki, R. (2017, June). A version of IPOP-CMA-ES algorithm with midpoint for CEC 2017 single objective bound constrained problems. In 2017 IEEE Congress on Evolutionary Computation (CEC) (pp. 1489- 1494). IEEE
2017
-
[31]
I., & Douglas, S
Amari, S. I., & Douglas, S. C. (1998, May). Why natural gradient?. In Proceedings of the 1998 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP’98 (Cat. No. 98CH36181) (V ol. 2, pp. 1213-1216). IEEE
1998
-
[32]
(2011, July)
Schaul, T., Glasmachers, T., & Schmidhuber, J. (2011, July). High di- mensions and heavy tails for natural evolution strategies. In Proceedings of the 13th annual conference on Genetic and evolutionary computation (pp. 845-852)
2011
-
[33]
Rudolph, G. (1997). Local convergence rates of simple evolutionary algorithms with Cauchy mutations. IEEE Transactions on Evolutionary Computation, 1(4), 249-258
1997
-
[34]
Ali, M., & Pant, M. (2011). Improving the performance of differential evolution algorithm using Cauchy mutation. Soft Computing, 15, 991- 1007
2011
-
[35]
Kappler, C. (1996). Are evolutionary algorithms improved by large mu- tations?. In Parallel Problem Solving from Nature—PPSN IV: Interna- tional Conference on Evolutionary Computation—The 4th International Conference on Parallel Problem Solving from Nature Berlin, Germany, Sep...
1996
-
[36]
J., Ahn, C
Choi, T. J., Ahn, C. W., & An, J. (2013). An adaptive Cauchy differential evolution algorithm for global numerical optimization. The Scientific World Journal, 2013
2013
-
[37]
L., Nguyen, D., & Nguyen, D
Kirkby, J. L., Nguyen, D., & Nguyen, D. (2019). Moments of Student’s t-distribution: A unified approach. arXiv preprint arXiv:1912.01607
2019 arXiv
-
[38]
Roth, M. (2012). On the multivariate t distribution. Link ¨oping University Electronic Press
2012
-
[39]
G., & Joarder, A
Kibria, B. G., & Joarder, A. H. (2006). A short review of multivariate t-distribution. Journal of Statistical research, 40(1), 59-72
2006
-
[40]
Kotz, S., & Nadarajah, S. (2004). Multivariate t-distributions and their applications. Cambridge University Press
2004
-
[41]
Akimoto, Y ., Nagata, Y ., Ono, I., & Kobayashi, S. (2012). Theoretical foundation for CMA-ES from information geometry perspective. Algo- rithmica, 64, 698-716. 11 APPENDIX We will give the derivation of the equation 17-19 p(x) = Γ( v 2 + d 2 ) Γ( v 2 ) ∗ |Σ|− 1 2 (v∗π) d 2 ∗...
2012
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.