REVIEW 4 major objections 6 minor 37 references
Large Language Model Enhanced Particle Swarm Optimization for Hyperparameter Tuning for Deep Learning Models
T0 review · 4 major / 6 minor · reviewed 2026-08-16 · deepseek-v4-flash
Pith's one-line read This paper claims that occasionally replacing PSO's worst particles with LLM-suggested positions cuts deep-learning hyperparameter-tuning model calls by 20–60 percent with no loss in accuracy or error.
desk verdict Promising hybrid idea, but the headline 20–60% model-call reduction is unsupported because the baseline runs a fixed 10 iterations while the LLM variant stops at the baseline's final cost—no random-replacement control. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is LLM-driven particle replacement, Algorithm 2: after initial PSO iterations, the LLM receives a prompt containing particle positions, velocities, and cost values; it returns an equal number of new positions; and the swarm replaces its worst particles with the best suggestions. This introduces externally generated diversity and guidance into the swarm's search while leaving the standard PSO velocity and position update equations intact. The paper's claimed effect—fewer model evaluations at equal cost—is carried by this replacement step.
What would settle it
Run the same Algorithm 2 but replace the LLM block with random samples from the search bounds (or reinitialized PSO particles), keeping the same prompt format, early-stopping target (the PSO-only cost), and evaluation budget; if random replacement matches the 20–60 percent reduction in model calls, the LLM-specific contribution is not confirmed.
Extended reading notes
Core claim
On its own terms, the paper's central claim is that injecting LLM-generated candidate positions into a running PSO swarm accelerates convergence without degrading the final model. The proposed algorithm runs standard PSO for a small number of iterations, sends the current particle positions, velocities, and costs to ChatGPT-3.5 or Llama3, receives an equal number of candidate positions, and replaces the worst-performing PSO particles with the best LLM suggestions; this loop repeats until the global best stops changing or the iteration budget is exhausted. In the LSTM regression case, PSO alone needed 50 model calls to reach the target cost while the LLM-enhanced version needed 20 calls with ChatGPT-3.5 and 30–40 with Llama3; in the CNN classification case both LLMs reduced calls from 50 to 20 at comparable accuracy. On the Rastrigin benchmark the savings are smaller (up to about 8.5 percent fewer iterations with Llama3), which the paper reads as evidence the method helps most when model evaluations are expensive.
Load-bearing premise
The reported speedup is measured against PSO-only runs with no control where worst particles are replaced by random or fresh PSO particles under identical early-stopping, so part or all of the savings could come from replacement diversity or the stopping rule rather than from the LLM's suggestions.
Editorial extensions
If this is right
- If the reported savings generalize, LLM queries can substitute for some deep-learning training calls, which matters because a single model evaluation can cost minutes or hours.
- The same replacement rule could be applied to other population-based metaheuristics; the paper explicitly suggests genetic-algorithm parent selection and mutation as natural targets.
- The method's efficiency claim implies that hyperparameter tuning can move onto resource-constrained settings such as IoT and edge devices, a direction the authors name.
- Because the savings differ by LLM (ChatGPT-3.5 saved 60 percent in regression; Llama3 saved 20–40 percent), the choice of LLM and prompt becomes a tunable component of the optimizer's cost.
- If the stopping rule is kept fixed, the 20–60 percent reduction in model calls translates directly into a training-budget reduction for comparable accuracy or RMSE.
Reading between the lines
- The prompt includes the current cost values of particles. A testable extension is to remove cost values from the prompt; if the savings vanish, the LLM is partly acting as a value-guided searcher rather than a pure generator of candidate architectures.
- A random-replacement control under the same early-stopping protocol would isolate how much of the savings comes from the LLM's content versus from the diversity introduced by any replacement; the paper does not run this control.
- The regression gains (60 percent with ChatGPT-3.5) are larger than the Rastrigin gains (up to 8.5 percent), which suggests the benefit scales with evaluation cost; one could test this by applying the method to progressively more expensive model families.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes LLM-driven PSO, a hybrid hyperparameter optimization method that replaces poorly performing PSO particles with positions suggested by ChatGPT-3.5 or Llama3. The method is evaluated on the Rastrigin benchmark, on LSTM-based AQI regression, and on CNN-based material classification. The authors report a 20% to 60% reduction in the number of deep learning model calls required to reach a target cost while maintaining comparable accuracy and error rates.
Significance. If the claimed reductions are real, the method would offer a practical way to cut the cost of hyperparameter search in deep learning, and the paper's idea of coupling LLM suggestions with swarm updates is timely and of interest to the optimization and AutoML communities. However, the experimental evidence as presented does not yet support the attribution of the savings to the LLM component, because the comparison protocol differs between the baseline and the proposed method and because key controls are missing. The paper does test on three scenarios, uses two different LLMs, and reports confidence intervals for the deep learning results, which is a useful level of transparency; nevertheless, the central claim of a 20% to 60% reduction in model calls is not yet established.
major comments (4)
- [Section IV-B and Tables 5-6] The reported 20% to 60% reduction in model calls is confounded by an asymmetric stopping protocol. The PSO baseline is always run for a fixed 10 iterations (50 model calls with 5 particles), whereas the LLM-driven variant is stopped as soon as its cost 'matched or surpassed that of PSO alone' (Section IV-B), which in Table 5 requires only 4 iterations (20 calls) for ChatGPT-3.5. This design cannot separate the effect of the LLM suggestions from the effects of early stopping and of the additional diversity introduced by replacing particles. A control in which the worst particles are replaced by random samples or by freshly initialized PSO particles under the same early-stopping rule is needed to attribute the savings to the LLM.
- [Section V-A, Tables 2 and 3] The Rastrigin results undercut the general convergence-speed claim stated in the abstract and conclusion. For 20 particles with balanced coefficients (c1=c2), standard PSO converges in a mean of 168.4 iterations, while ChatGPT-3.5-driven PSO takes 173.9 iterations (Tables 2 and 3). The 20% to 60% reduction in model calls is observed only in the deep learning experiments, which are the ones lacking the control described above. Without a demonstrated advantage on a standard benchmark, the claim that LLM suggestions accelerate PSO is not generally supported.
- [Section V-B and V-C, Tables 5 and 6] The statistical support is weak: each method was run only three times, and the 95% confidence intervals overlap substantially. For the classification task, PSO accuracy is 0.8626 with CI (0.8601, 0.8652), while ChatGPT-3.5 and Llama3 give CIs (0.8442, 0.8640) and (0.8480, 0.8612), respectively, so the claim of comparable accuracy is not statistically established. Similarly, the RMSE intervals for the regression task overlap. Additional repetitions or a formal equivalence test are needed before the 'preserving accuracy and error rates' claim can be accepted.
- [Section III-C and Section IV prompt] The prompt fed to the LLM includes the current cost values for every particle (the fifth entry of the particle string), and the LLM is asked to output new positions that reduce the cost. Because the LLM sees the exact objective values, the improvement could simply reflect a response to the presented costs rather than a transferable search heuristic. The paper provides no ablation that removes or masks the cost information, so the contribution of the LLM's generative capabilities versus its use of the cost vector remains untested.
minor comments (6)
- [Table 1] The layer velocity entry 'Min = 1, Max = 0.2(MaxLayers - MinLayers)' is ambiguous; it should clarify whether these are the minimum and maximum bounds for the random initialization of velocities or a formula for a velocity range.
- [Figures 6 and 8] The annotations '20 %' and '60 %' on the model-call bars are not defined in the captions; please indicate that they refer to the percentage reduction relative to the PSO baseline.
- [Section VI] The lesson that 'reducing the initial PSO iterations at each step resulted in earlier convergence' is not supported by Table 7, where the number of iterations is non-monotonic (for example, with 20 particles it moves from 123 to 140 to 147 to 168 to 165 as the initial PSO horizon increases).
- [Abstract and Introduction] The term 'groundbreaking methodology' in the abstract is subjective and should be replaced with a neutral description of the proposed approach.
- [Section VI] The statement that 'Our method consistently converged within four iterations' is inconsistent with Table 5, where Llama3 uses 6 to 8 iterations in the regression task.
- [Section IV prompt] The prompt template lists Npop values 5, 10, 15, 20, 50, or 100, but Table 4 only reports 5, 10, 15, 20 for the deep learning experiments; please reconcile the numbers.
Circularity Check
No significant circularity: the paper's claims are empirical comparisons against an external PSO baseline, not derivations that reduce to their own inputs.
full rationale
The paper proposes LLM-enhanced PSO and evaluates it empirically on Rastrigin, LSTM regression, and CNN classification. There is no derivation chain in which a predicted quantity is defined in terms of the fitted or cited input, and no claimed result is equivalent to an input by construction. The central speedup claim is an experimental comparison of model-call counts between PSO and LLM-assisted PSO, and the paper reports the stopping protocol explicitly: in the regression and classification cases, the LLM variant stops when its cost 'matched or surpassed that of PSO alone' after fewer iterations. This is an experimental-design concern about confounded comparison (asymmetric stopping targets, no random-replacement control), not a circularity of the kind defined by the review criteria. A fairness concern about whether the improvement is attributable to the LLM belongs to correctness risk, not to circularity. The Rastrigin tables even show cases where LLM-driven PSO requires more iterations than PSO, which undermines any claim that the improvement is definitionally forced. Self-citations appear only as related-work context and are not load-bearing for the paper's proposed method. Therefore, no circular step is identified and the circularity score is 0.
Assumptions & free parameters
free parameters (3)
- Initial PSO iterations before first LLM query =
2 for deep learning tasks; 10 to 50 explored for Rastrigin
- Target cost stopping threshold =
RMSE 0.1343 for regression; accuracy 0.8622 for classification
- Population size =
5 for deep learning; 20, 50, 100 for Rastrigin
assumptions (4)
- standard math The standard PSO velocity-position updates (Equations 1 to 4) are a valid baseline for hyperparameter search.
- domain assumption LLM outputs, when prompted with particle positions and costs, can be treated as useful optimizer suggestions.
- domain assumption The selected LSTM and CNN models and the AQI and recyclable/organic datasets are representative deep learning hyperparameter optimization tasks.
- ad hoc to paper The prompt's inclusion of cost values does not forfeit the LLM's suggestions.
Cite this review
Pith. "Pith review of Large Language Model Enhanced Particle Swarm Optimization for Hyperparameter Tuning for Deep Learning Models." pith.science (2026). https://pith.science/paper/RYL7RQII
@misc{pith2026250414126,
author = {Pith},
title = {Pith review of: Large Language Model Enhanced Particle Swarm Optimization for Hyperparameter Tuning for Deep Learning Models},
year = {2026},
howpublished = {\url{https://pith.science/paper/RYL7RQII}},
note = {Machine review of arXiv:2504.14126}
}
read the original abstract
Determining the ideal architecture for deep learning models, such as the number of layers and neurons, is a difficult and resource-intensive process that frequently relies on human tuning or computationally costly optimization approaches. While Particle Swarm Optimization (PSO) and Large Language Models (LLMs) have been individually applied in optimization and deep learning, their combined use for enhancing convergence in numerical optimization tasks remains underexplored. Our work addresses this gap by integrating LLMs into PSO to reduce model evaluations and improve convergence for deep learning hyperparameter tuning. The proposed LLM-enhanced PSO method addresses the difficulties of efficiency and convergence by using LLMs (particularly ChatGPT-3.5 and Llama3) to improve PSO performance, allowing for faster achievement of target objectives. Our method speeds up search space exploration by substituting underperforming particle placements with best suggestions offered by LLMs. Comprehensive experiments across three scenarios -- (1) optimizing the Rastrigin function, (2) using Long Short-Term Memory (LSTM) networks for time series regression, and (3) using Convolutional Neural Networks (CNNs) for material classification -- show that the method significantly improves convergence rates and lowers computational costs. Depending on the application, computational complexity is lowered by 20% to 60% compared to traditional PSO methods. Llama3 achieved a 20% to 40% reduction in model calls for regression tasks, whereas ChatGPT-3.5 reduced model calls by 60% for both regression and classification tasks, all while preserving accuracy and error rates. This groundbreaking methodology offers a very efficient and effective solution for optimizing deep learning models, leading to substantial computational performance improvements across a wide range of applications.
Figures
Figures from the paper (6 more)
Reference graph
Works this paper leans on
-
[1]
Grid search-based hyperparameter tuning and classification of microarray cancer data,
B. Shekar and G. Dagnew, “Grid search-based hyperparameter tuning and classification of microarray cancer data,” in 2019 second inter- national conference on advanced computational and communication paradigms (ICACCP). IEEE, 2019, pp. 1–8
work page 2019
-
[2]
D. M. Belete and M. D. Huchaiah, “Grid search in hyperparameter optimization of machine learning models for prediction of HIV/AIDS test results,” International Journal of Computers and Applications , vol. 44, no. 9, pp. 875–886, 2022
work page 2022
-
[3]
Distributed tuning of machine learning algorithms using MapReduce clusters,
Y . Ganjisaffar, T. Debeauvais, S. Javanmardi, R. Caruana, and C. V . Lopes, “Distributed tuning of machine learning algorithms using MapReduce clusters,” in Proceedings of the Third Workshop on Large Scale Data Mining: Theory and Applications , 2011, pp. 1–8
work page 2011
-
[4]
Comparison of optimization techniques based on gradient descent algorithm: A review,
S. H. Haji and A. M. Abdulazeez, “Comparison of optimization techniques based on gradient descent algorithm: A review,” PalArch’s Journal of Archaeology of Egypt/Egyptology, vol. 18, no. 4, pp. 2715– 2743, 2021
work page 2021
-
[5]
An efficient optimization approach for designing machine learning models based on genetic algorithm,
K. M. Hamdia, X. Zhuang, and T. Rabczuk, “An efficient optimization approach for designing machine learning models based on genetic algorithm,” Neural Computing and Applications , vol. 33, no. 6, pp. 1923–1933, 2021
work page 1923
-
[6]
Particle swarm optimization: A com- prehensive survey,
T. M. Shami, A. A. El-Saleh, M. Alswaitti, Q. Al-Tashi, M. A. Summakieh, and S. Mirjalili, “Particle swarm optimization: A com- prehensive survey,” Ieee Access, vol. 10, pp. 10 031–10 061, 2022
work page 2022
-
[7]
Genetic algorithms in wireless networking: techniques, applications, and issues,
U. Mehboob, J. Qadir, S. Ali, and A. Vasilakos, “Genetic algorithms in wireless networking: techniques, applications, and issues,” Soft Computing, vol. 20, pp. 2467–2501, 2016
work page 2016
-
[8]
Particle swarm optimization,
J. Kennedy and R. Eberhart, “Particle swarm optimization,” in Pro- ceedings of ICNN’95-international conference on neural networks , vol. 4. ieee, 1995, pp. 1942–1948
1995
Show all 37 references
-
[9]
A review of metaheuristic techniques for optimal integration of electrical units in distribution networks,
K. E. Adetunji, I. W. Hofsajer, A. M. Abu-Mahfouz, and L. Cheng, “A review of metaheuristic techniques for optimal integration of electrical units in distribution networks,” IEEE Access , vol. 9, pp. 5046–5068, 2020
2020
-
[10]
A comparative study of genetic algorithm and the particle swarm optimization,
S. Katiyar, “A comparative study of genetic algorithm and the particle swarm optimization,” International Journal of Technology , vol. 2, no. 2, pp. 21–24, 2010
2010
-
[11]
Comparison of three evolutionary algorithms: GA, PSO, and DE,
V . Kachitvichyanukul, “Comparison of three evolutionary algorithms: GA, PSO, and DE,” Industrial Engineering and Management Systems, vol. 11, no. 3, pp. 215–223, 2012
2012
-
[12]
A comparison of extremal optimization, differential evolution and particle swarm optimization methods for well placement design in groundwater management,
F. Redoloza and L. Li, “A comparison of extremal optimization, differential evolution and particle swarm optimization methods for well placement design in groundwater management,” Mathematical Geosciences, vol. 53, no. 4, pp. 711–735, 2021
2021
-
[13]
Inertia weight strategies in particle swarm optimization,
J. C. Bansal, P. Singh, M. Saraswat, A. Verma, S. S. Jadon, and A. Abraham, “Inertia weight strategies in particle swarm optimization,” in 2011 Third world congress on nature and biologically inspired computing. IEEE, 2011, pp. 633–640
2011
-
[14]
Language models are unsupervised multitask learners,
A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, I. Sutskever et al., “Language models are unsupervised multitask learners,” OpenAI blog, vol. 1, no. 8, p. 9, 2019
2019
-
[15]
Language mod- els are few-shot learners,
T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askellet al., “Language mod- els are few-shot learners,” Advances in neural information processing systems, vol. 33, pp. 1877–1901, 2020
1901
-
[16]
Hyper- parameter optimization of convolutional neural network based on par- ticle swarm optimization algorithm,
Z. Fouad, M. Alfonse, M. Roushdy, and A.-B. M. Salem, “Hyper- parameter optimization of convolutional neural network based on par- ticle swarm optimization algorithm,” Bulletin of Electrical Engineering and Informatics, vol. 10, no. 6, pp. 3377–3384, 2021
2021
-
[17]
PSO based Hyperparameter tuning of CNN Multivariate Time-Series Anal- ysis,
A. B. P. Utama, A. P. Wibawa, M. Muladi, and A. Nafalski, “PSO based Hyperparameter tuning of CNN Multivariate Time-Series Anal- ysis,” Jurnal Online Informatika , vol. 7, no. 2, pp. 193–202, 2022
2022
-
[18]
SVM hyper-parameters optimization using multi-PSO for intrusion detection,
D. J. Kalita, V . P. Singh, and V . Kumar, “SVM hyper-parameters optimization using multi-PSO for intrusion detection,” in Social Net- working and Computational Intelligence: Proceedings of SCI-2018 . Springer, 2020, pp. 227–241
2018
-
[19]
Deep neural networks hyperparameter optimization using particle swarm optimization for detecting frauds transactions,
M. Tayebi and S. El Kafhali, “Deep neural networks hyperparameter optimization using particle swarm optimization for detecting frauds transactions,” in Advances on Smart and Soft Computing: Proceedings of ICACIn 2021 . Springer, 2022, pp. 507–516
2021
-
[20]
Parameters optimization of deep learning models using particle swarm optimization,
B. Qolomany, M. Maabreh, A. Al-Fuqaha, A. Gupta, and D. Benhad- dou, “Parameters optimization of deep learning models using particle swarm optimization,” in 2017 13th International Wireless Communi- cations and Mobile Computing Conference (IWCMC) . IEEE, 2017, pp. 1285–1290
2017
-
[21]
Particle swarm optimized federated learning for industrial iot and smart city services,
B. Qolomany, K. Ahmad, A. Al-Fuqaha, and J. Qadir, “Particle swarm optimized federated learning for industrial iot and smart city services,” in GLOBECOM 2020 - 2020 IEEE Global Communications Conference, 2020, pp. 1–6
2020
-
[22]
Enhancing per- sonalized learning with explainable ai: A chaotic particle swarm optimization based decision support system,
R. Parkavi, P. Karthikeyan, and A. S. Abdullah, “Enhancing per- sonalized learning with explainable ai: A chaotic particle swarm optimization based decision support system,” Applied Soft Computing, vol. 156, p. 111451, 2024
2024
-
[23]
Particle swarm optimization-based hyperparameters tuning of machine learning mod- els for big covid-19 data analysis,
H. S. Salem, M. A. Mead, and G. S. El-Taweel, “Particle swarm optimization-based hyperparameters tuning of machine learning mod- els for big covid-19 data analysis,” Journal of Computer and Commu- nications, vol. 12, no. 3, pp. 160–183, 2024
2024
-
[24]
Optimized three deep learning models based- pso hyperparameters for beijing pm2. 5 prediction,
A. Pranolo, Y . Mao, A. P. Wibawa, A. B. P. Utama, and F. A. Dwiyanto, “Optimized three deep learning models based- pso hyperparameters for beijing pm2. 5 prediction,” arXiv preprint arXiv:2306.07296, 2023
2023 arXiv
-
[25]
Large language models as optimizers,
C. Yang, X. Wang, Y . Lu, H. Liu, Q. V . Le, D. Zhou, and X. Chen, “Large language models as optimizers,” 2024. [Online]. Available: https://arxiv.org/abs/2309.03409
2024 arXiv
-
[26]
Are large language models good prompt optimizers?
R. Ma, X. Wang, X. Zhou, J. Li, N. Du, T. Gui, Q. Zhang, and X. Huang, “Are large language models good prompt optimizers?” arXiv preprint arXiv:2402.02101 , 2024. :
2024 arXiv
-
[27]
Large language models are human-level prompt engineers,
Y . Zhou, A. I. Muresanu, Z. Han, K. Paster, S. Pitis, H. Chan, and J. Ba, “Large language models are human-level prompt engineers,” arXiv preprint arXiv:2211.01910 , 2022
2022 arXiv
-
[28]
MyCrunchGPT: A LLM assisted framework for scientific machine learning,
V . Kumar, L. Gleyzer, A. Kahana, K. Shukla, and G. E. Karniadakis, “MyCrunchGPT: A LLM assisted framework for scientific machine learning,” Journal of Machine Learning for Modeling and Computing , vol. 4, no. 4, 2023
2023
-
[29]
Large language model agent for hyper- parameter optimization,
S. Liu, C. Gao, and Y . Li, “Large language model agent for hyper- parameter optimization,” arXiv preprint arXiv:2402.01881 , 2024
2024 arXiv
-
[30]
Using large language models for hyperparameter optimization,
M. R. Zhang, N. Desai, J. Bae, J. Lorraine, and J. Ba, “Using large language models for hyperparameter optimization,” in NeurIPS 2023 Foundation Models for Decision Making Workshop , 2023
2023
-
[31]
Hyperparameter tuning of load-forecasting models using metaheuristic optimization algo- rithms—a systematic review,
U. Mumtahina, S. Alahakoon, and P. Wolfs, “Hyperparameter tuning of load-forecasting models using metaheuristic optimization algo- rithms—a systematic review,” Mathematics, vol. 12, no. 21, p. 3353, 2024
2024
-
[32]
ChatGPT for good? On opportunities and challenges of large language models for education,
E. Kasneci, K. Seßler, S. K ¨uchemann, M. Bannert, D. Dementieva, F. Fischer, U. Gasser, G. Groh, S. G ¨unnemann, E. H ¨ullermeier et al., “ChatGPT for good? On opportunities and challenges of large language models for education,” Learning and individual differences , vol. 103...
2023
-
[33]
Study and analysis of chat GPT and its impact on different fields of study,
D. Kalla, N. Smith, F. Samaah, and S. Kuraku, “Study and analysis of chat GPT and its impact on different fields of study,” International journal of innovative science and research technology , vol. 8, no. 3, 2023
2023
-
[34]
A brief overview of ChatGPT: The history, status quo and potential future development,
T. Wu, S. He, J. Liu, S. Sun, K. Liu, Q.-L. Han, and Y . Tang, “A brief overview of ChatGPT: The history, status quo and potential future development,” IEEE/CAA Journal of Automatica Sinica, vol. 10, no. 5, pp. 1122–1136, 2023
2023
-
[35]
The Llama 3 Herd of Models,
A. Dubey, A. Jauhri, A. Pandey, A. Kadian, A. Al-Dahle, A. Letman, A. Mathur, A. Schelten, A. Yang, A. Fan et al., “The Llama 3 Herd of Models,” arXiv preprint arXiv:2407.21783 , 2024
2024 arXiv
-
[36]
How good are low-bit quantized LLaMA3 models? an empirical study,
W. Huang, X. Ma, H. Qin, X. Zheng, C. Lv, H. Chen, J. Luo, X. Qi, X. Liu, and M. Magno, “How good are low-bit quantized LLaMA3 models? an empirical study,” arXiv preprint arXiv:2404.14047 , 2024
2024 arXiv
-
[37]
Deep learning based multimodal urban air quality prediction and traffic analytics,
S. Hameed, A. Islam, K. Ahmad, S. B. Belhaouari, J. Qadir, and A. Al- Fuqaha, “Deep learning based multimodal urban air quality prediction and traffic analytics,” Scientific Reports, vol. 13, no. 1, p. 22181, 2023. Saad Hameed received his B.Sc. in Electrical Engineering from ...
2016
Reviewed August 16, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.