Pith. sign in

REVIEW 3 major objections 6 minor 45 references

LiBOG: Lifelong Learning for Black-Box Optimizer Generation

T0 review · 3 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read LiBOG claims that lifelong reinforcement learning can generate black-box optimizers for a sequence of arriving problem distributions, keeping performance on earlier distributions nearly intact while still learning new ones.

desk verdict A useful first step for lifelong MetaBBO with a plausible consolidation recipe, but the quantitative ranking claims are not backed by significance testing. read the letter →

arxiv 2505.13025 v1 pith:THFKPEWM submitted 2025-05-19 cs.LG cs.AI

Jiyuan Pei , Yi Mei , Jialin Liu , Mengjie Zhang This is my paper · ORCID
classification cs.LGcs.AI
keywords lifelonglearningcontinualmetablack-boxoptimizationoptimizergenerationcatastrophicforgettingsymbolicequationreinforcementelasticweightconsolidation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Meta-black-box optimization has so far trained a model once on a fixed distribution of problems and then frozen it, but real optimization workloads arrive as a stream of related, shifting distributions. LiBOG treats each arriving distribution as a task in a non-stationary Markov decision process and keeps training one policy that emits symbolic solution-update rules, so a single model serves every distribution it has seen. It combines two consolidation mechanisms—elastic weight consolidation across tasks and a new elite-behavior consolidation within tasks—to fight catastrophic forgetting while preserving plasticity. On four benchmark problem categories arranged in three different task orders, the paper reports that LiBOG ranks first in eight of twelve task-position cases and that removing either consolidation term significantly hurts performance. The payoff, if true, is that automated optimizer design can keep improving as new problem distributions appear instead of being retrained from scratch or frozen after one training run.

What carries the argument

The load-bearing object is the composite loss $L = L_{\mathrm{PPO}} + \alpha\,CL_{\mathrm{inter}} + \beta\,CL_{\mathrm{intra}}$, where $CL_{\mathrm{inter}}$ is the EWC penalty built from per-task importance matrices $\Omega_j$ and stored task parameters $\theta_j^*$, and $CL_{\mathrm{intra}}$ is the KL penalty that pulls the current policy toward an elite policy $\pi_{\theta_e}$ on observed states. The surrounding machinery is the non-stationary MDP formulation that turns each arriving problem distribution into a task, the LSTM that generates symbolic update rules, and the elite-update rule that copies the current model whenever its average final objective beats the stored elite. This combination lets the model preserve old behaviors without storing raw experience.

What would settle it

A replication on the same three task orders would falsify the central claim if, after training the full sequence, LiBOG's normalized performance on an early task fell below that of fine-tuning on the same task, or if removing either consolidation term left final performance unchanged. A sharper check is to run the same training protocol with hidden task boundaries or gradual changes in the benchmark's offset and rotation parameters; if forgetting returns and LiBOG no longer beats fine-tuning, the method's applicability is limited to the clean-block regime it assumes.

Watch

Extended reading notes

Core claim

The central discovery is that catastrophic forgetting in meta-black-box optimization can be controlled by pairing parameter-space and behavior-space consolidation in a single lifelong reinforcement-learning objective. LiBOG models lifelong learning as a non-stationary MDP built from stationary MDPs, one per problem distribution, with a shared state space of fitness-landscape metrics and a shared action space of symbolic update trees. An LSTM policy trained with PPO produces the tree-structured rules for each optimization step. After each task, the method stores the task's parameters and a per-parameter importance matrix and adds the EWC penalty $CL_{\mathrm{inter}}$ (Eq. 3) to all later losses; during a task, it maintains an elite policy—the best-performing model seen so far by average final objective—and penalizes the KL divergence between current and elite action distributions via $CL_{\mathrm{intra}}$ (Eq. 4). The paper demonstrates on the CEC uni-modal, basic, hybrid, and composition categories that with both terms the test performance on earlier tasks stays almost flat after later training, that new tasks are still learned well, and that both consolidation mechanisms contribute to the final performance.

Load-bearing premise

LiBOG assumes the identity and arrival boundary of each problem distribution is known, so every task is a clean stationary block and can be stored as a separate parameter-plus-importance record; if distributions drift continuously or switch without labels, the method cannot be applied as described.

Editorial extensions

If this is right

  • A single optimizer-generation model can be deployed across a growing set of problem distributions, so new arrivals do not force a freeze-and-discard retraining cycle.
  • Because only parameters and importance matrices are stored per task, memory overhead stays low ($O(|\theta|\cdot I)$) rather than requiring large experience buffers.
  • Starting each new task from transferred knowledge gives LiBOG usable performance on a new distribution before it has been trained on it, an effect the paper documents at task boundaries.
  • Both consolidation terms are necessary: the reported ablation shows that removing either one lowers average normalized performance, with the intra-task term having the larger effect in that comparison.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same EWC-plus-elite-behavior recipe does not depend on symbolic rules: it could be grafted onto other MetaBBO designs, such as end-to-end solution generators or parameter-configuration policies, because it acts on any policy's parameters and action distributions.
  • If task boundaries were unknown or distributions drifted smoothly, LiBOG's stored-task bookkeeping would need to be replaced by online change-point detection or a generative model of tasks; the known-boundary assumption marks that as the next regime to test.
  • Because the reported normalized scores are compressed (the supplement's raw-objective example shows LiBOG at 2945.91 versus 4354.64 on the final task of one order), small normalized differences may correspond to large practical gaps; re-reporting on raw objective values would make the comparison easier to interpret.
  • A robustness extension would replace the elite-update criterion of average final objective with a robust statistic such as median or worst-case performance, since black-box runs are noisy; whether EBC's benefit survives noisier reward estimates is not quantified in the paper.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper introduces LiBOG, a lifelong learning framework for meta-black-box optimization (MetaBBO). The setting is a sequence of tasks, each defined by a problem distribution; the goal is a single model that generates symbolic solution-update rules (optimizers) that work well on all previously seen distributions. LiBOG builds on SYMBOL's PPO-based symbolic equation learning and adds two consolidation mechanisms: inter-task elastic weight consolidation (EWC; Eq. 3) and an intra-task elite behavior consolidation (EBC; Eq. 4) that regularizes the current policy toward an elite policy by KL divergence. Experiments on three task orders constructed from CEC benchmark function categories compare LiBOG against restart, fine-tuning, all-task training, and MadDE, reporting ranks, forgetting curves, sensitivity analysis, and an ablation study. The paper claims that LiBOG mitigates catastrophic forgetting while retaining plasticity, is ranked first in eight of twelve task cases, and is robust across task orders and hyper-parameter settings.

Significance. If the claims hold, this is one of the first lifelong learning approaches for MetaBBO, addressing a practically important limitation of current one-off MetaBBO training. The idea of consolidating both inter-task and intra-task knowledge is sensible, and the qualitative forgetting curves (Figure 3) suggest that the proposed consolidation terms do reduce catastrophic forgetting relative to fine-tuning. The paper also ships code, and the ablation and sensitivity studies give useful information about the method's behavior. However, the quantitative evidence for the headline claims is currently weak: the normalized score differences between LiBOG and baselines are often smaller than the reported run-to-run standard deviations, and no significance testing is provided. In addition, the mathematical definitions of the consolidation terms contain inconsistencies and unspecified computational details that are central to the method. These issues are fixable but require substantive revisions.

major comments (3)
  1. [Section 3.3, Eqs. (2)-(3)] The storage complexity claim for inter-task consolidation is inconsistent with the equations. Eq. (2) defines Ω_i as an outer product ∇ℓ∇ℓ^T, which is a full d×d matrix, and Eq. (3) uses Ω_j inside a quadratic form (θ−θ*_j)^T Ω_j (θ−θ*_j). This would require O(|θ|^2) storage per task and O(|θ|^2 I) overall, contradicting the stated O(|θ| I) space complexity. If the implementation actually uses a diagonal approximation (as is standard in EWC), the authors must state this explicitly and define Ω_i as the diagonal of the outer-product matrix (or as the average squared gradient), and they must clarify how Eq. (3) is computed under that approximation.
  2. [Section 3.4, Eq. (4)] Equation (4) defines the intra-task consolidation loss as a sum over all actions a∈A of π_θ(a|s) log(π_θ(a|s)/π_θe(a|s)). In this paper, an action is a complete tree-structured symbolic update rule generated autoregressively by an LSTM, so the action space is exponentially large (or effectively unbounded). The full sum over A is therefore intractable as written. The paper must specify how CL_intra is actually computed in practice: for example, whether it uses a per-step categorical KL divergence over the node vocabulary, a Monte Carlo estimate from the sampled trajectories, or a closed-form expression exploiting the specific policy parametrization. Without this detail the method is not fully specified and cannot be reproduced.
  3. [Section 4.1 and Tables 4-6] The headline ranking claim that LiBOG is first in eight of twelve cases is based on mean normalized scores whose differences are often smaller than the reported standard deviations. For instance, in Table 4 (order 0), LiBOG on P0 is 0.999138±0.000499 vs fine-tuning 0.998749±0.001087 (difference 0.000389), and on P1 is 0.925060±0.004585 vs 0.924385±0.008710 (difference 0.000675), both well within one standard deviation of each other. No p-values, confidence intervals, or effect sizes are reported anywhere. Similarly, in the ablation (Table 2), the difference between LiBOG (0.982440) and only-intra (0.982306) is 0.000134 over five runs, yet the text states that removing either component 'significantly reduces' performance. The authors should add appropriate inferential statistics (e.g., paired tests across the 10 runs, or confidence intervals on the ranks) or substantially soften the quantitative comparative claims. The forgetting curves in Figure 3 provide qualitative support for the method's benefit on early tasks, but they do not justify the 'eight of twelve' statement.
minor comments (6)
  1. [Table 1] The layout of Table 1 is garbled and very hard to interpret; the column grouping under each task order is unclear, and the rank entries for restart and all-task appear to overlap across orders. Please reformat the table so each task order is clearly separated and the values are unambiguous.
  2. [Appendix A] There is a typo: 'the potion of remaining optimization iterations' should be 'the portion of remaining optimization iterations'.
  3. [Table 5] The baseline name 'fintuning' should be 'fine-tuning'.
  4. [Section 4.3] The sentence in the ablation study, 'removing either significantly reduces its performance,' is not supported by the reported numbers or any statistical test; please rephrase to reflect the magnitude of the observed differences or add a test.
  5. [Section 4.3 and Table 8] The text says 'the best-performance weight values above are used' for the ablation, but the LiBOG row in Table 8 (e.g., P0=0.999528, P3=0.994326) does not match any row in Table 7 (e.g., α=10,β=1 gives P0=0.999714, P3=0.993138). Please clarify which weight setting was used and why the numbers differ.
  6. [Section 4.1] The statement that 'LiBOG outperforms all baseline methods for each of the three task orders' is too strong given that restart achieves the best performance in some individual cases (as acknowledged later). Please rephrase to indicate that LiBOG has the best average rank per order, or otherwise clarify the comparison.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: LiBOG's claims rest on empirical benchmarks and standard regularizers, not on self-referential equations or fitted predictions.

full rationale

LiBOG's derivation chain is empirical and self-contained. Eq. (3) is the standard EWC quadratic penalty using importance estimates from Eq. (2), and Eq. (4) is a KL divergence behavior-regularization term; neither is defined in terms of the experimental outcomes it claims to explain. The paper's central claims, that the two consolidation terms mitigate forgetting and improve plasticity, are supported by comparisons against fine-tuning, restart, all-task, and MadDE on sampled problems from the same task distributions. Those comparisons are ordinary empirical evaluation, not a derivation of the result from its inputs. The hyper-parameters alpha and beta are set to 1 as a rule of thumb, and the weight-sweep/ablation uses the first task order; this is model selection, not a fitted parameter being renamed as a prediction. Self-citations ([Pei et al., 2024], [Pei et al., 2025]) support background claims only and are not load-bearing. The acknowledged limitations (unknown task boundaries, continuously changing distributions, EBC constraining exploration) and the Appendix C note about the off-policy importance estimate are explicit statements of scope, not hidden circularity. The absence of significance testing on near-equal means in Tables 4-6 is a statistical-rigor concern, not a circularity concern. No equation in the paper reduces to its own input, and no external 'uniqueness theorem' is imported.

Assumptions & free parameters 2 free parameters · 5 assumptions · 0 invented entities

The central claim rests on known-task-boundary assumptions, the MDP/FLA state representation, and the adopted SYMBOL backbone, plus the two hand-set loss weights alpha and beta. No new physical or ontological entities are introduced; the elite model in EBC is an algorithmic snapshot, not an independent entity.

free parameters (2)
  • alpha (inter-task consolidation weight) = 1 (rule of thumb; sensitivity grid 0.1, 1, 10 tested)
    Weight of CL_inter in Eq. (5); chosen by hand, not tuned on a validation set.
  • beta (intra-task consolidation weight) = 1 (rule of thumb; sensitivity grid 0.1, 1, 10 tested)
    Weight of CL_intra in Eq. (5); chosen by hand, not tuned on a validation set.
assumptions (5)
  • domain assumption Task identities and boundaries are known
    Section 3.1 explicitly assumes known task boundaries; EWC stores per-task parameter snapshots and importance matrices, so unknown or drifting boundaries break the method. The authors list this as a limitation.
  • domain assumption Each task is a stationary MDP sharing the same state space, action space, and iteration budget across tasks
    Section 3.1 defines the non-stationary MDP as a sequence of stationary MDPs with identical S, A, H; if tasks have different budgets or representations, the transfer and regularization assumptions need rework.
  • domain assumption Fitness landscape analysis metrics are a sufficient state representation for constructing update rules across tasks
    Section 3.1 and Appendix A adopt the nine FLA metrics from SYMBOL without evidence that they capture enough information for lifelong transfer across different function categories.
  • domain assumption The SYMBOL reward function is an appropriate learning signal
    Equation (1) is taken from Chen et al. [2024]; its bias toward the MadDE guide optimizer is not independently validated in this paper.
  • standard math PPO and EWC behave as assumed in this setting
    The paper relies on PPO [Schulman et al. 2017] and EWC [Kirkpatrick et al. 2017] as known-good algorithms without re-deriving them.

how reviews work

0 comments
Cite this review

Pith. "Pith review of LiBOG: Lifelong Learning for Black-Box Optimizer Generation." pith.science (2026). https://pith.science/paper/THFKPEWM

@misc{pith2026250513025,
  author       = {Pith},
  title        = {Pith review of: LiBOG: Lifelong Learning for Black-Box Optimizer Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/THFKPEWM}},
  note         = {Machine review of arXiv:2505.13025}
}
read the original abstract

Meta-Black-Box Optimization (MetaBBO) garners attention due to its success in automating the configuration and generation of black-box optimizers, significantly reducing the human effort required for optimizer design and discovering optimizers with higher performance than classic human-designed optimizers. However, existing MetaBBO methods conduct one-off training under the assumption that a stationary problem distribution with extensive and representative training problem samples is pre-available. This assumption is often impractical in real-world scenarios, where diverse problems following shifting distribution continually arise. Consequently, there is a pressing need for methods that can continuously learn from new problems encountered on-the-fly and progressively enhance their capabilities. In this work, we explore a novel paradigm of lifelong learning in MetaBBO and introduce LiBOG, a novel approach designed to learn from sequentially encountered problems and generate high-performance optimizers for Black-Box Optimization (BBO). LiBOG consolidates knowledge both across tasks and within tasks to mitigate catastrophic forgetting. Extensive experiments demonstrate LiBOG's effectiveness in learning to generate high-performance optimizers in a lifelong learning manner, addressing catastrophic forgetting while maintaining plasticity to learn new tasks.

Figures

Figures reproduced from arXiv: 2505.13025 by the authors.

Figure 1
Figure 1. Existing MetaBBO methods train the BBO optimizer [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Illustration of LiBOG, with different problem distributions [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Test performance on each learned task during the lifelong [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (4 more)
Figure 5
Figure 5. Figure 5: Performance of LiBOG trained with different values of [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Test performance on each learned task during the lifelong [PITH_FULL_IMAGE:figures/full_fig_p011_6.png]
Figure 7
Figure 7. Figure 7: Test performance on each learned task during the lifelong [PITH_FULL_IMAGE:figures/full_fig_p011_7.png]
Figure 8
Figure 8. Figure 8: Test performance of each model obtained during the life [PITH_FULL_IMAGE:figures/full_fig_p012_8.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

45 extracted references · 43 canonical work pages

  1. [1]

    Policy and value transfer in lifelong reinforcement learning

    [Abelet al., 2018 ] David Abel, Yuu Jinnai, Sophie Yue Guo, George Konidaris, and Michael Littman. Policy and value transfer in lifelong reinforcement learning. InInterna- tional Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Research, pages 20–29. PMLR, 10–15 Jul

    work page 2018

  2. [7]

    Hoffman, Ser- gio G ´omez Colmenarejo, Misha Denil, Timothy P

    [Chenet al., 2017 ] Yutian Chen, Matthew W. Hoffman, Ser- gio G ´omez Colmenarejo, Misha Denil, Timothy P. Lil- licrap, Matt Botvinick, and Nando de Freitas. Learn- ing to learn without gradient descent by gradient descent. InInternational Conference on Machine Learning, vol- ume 70 ofProceedings of Machine Learning Research, pages 748–756. PMLR, 06–11 Aug

    work page 2017

  3. [12]

    Optimizing large-scale hy- perparameters via automated learning algorithm.arXiv, 2102.09026,

    [Guet al., 2021 ] Bin Gu, Guodong Liu, Yanfu Zhang, Xi- ang Geng, and Heng Huang. Optimizing large-scale hy- perparameters via automated learning algorithm.arXiv, 2102.09026,

    arXiv 2021

  4. [13]

    Reinforcement learning for adaptive operator selec- tion in memetic search applied to quadratic assignment problem

    [Handokoet al., 2014 ] Stephanus Daniel Handoko, Duc Thien Nguyen, Zhi Yuan, and Hoong Chuin Lau. Reinforcement learning for adaptive operator selec- tion in memetic search applied to quadratic assignment problem. InCompanion Publication of Conference on Genetic and Evolutionary Computation, page 193–194. ACM,

    work page 2014

  5. [16]

    Transient non-stationarity and generalisation in deep rein- forcement learning

    [Iglet al., 2021 ] Maximilian Igl, Gregory Farquhar, Jelena Luketina, Wendelin Boehmer, and Shimon Whiteson. Transient non-stationarity and generalisation in deep rein- forcement learning. InInternational Conference on Learn- ing Representations,

    work page 2021

  6. [17]

    Towards continual reinforcement learning: A review and perspectives

    [Khetarpalet al., 2022 ] Khimya Khetarpal, Matthew Riemer, Irina Rish, and Doina Precup. Towards continual reinforcement learning: A review and perspectives. Journal of Artificial Intelligence Research, 75:1401–1476, December

    work page 2022

  7. [18]

    Kirkpatrick, C

    [Kirkpatricket al., 1983 ] S. Kirkpatrick, C. D. Gelatt, and M. P. Vecchi. Optimization by simulated annealing.Sci- ence, 220(4598):671–680,

    work page 1983

  8. [20]

    Rupam Mahmood

    [Lanet al., 2023 ] Qingfeng Lan, Yangchen Pan, Jun Luo, and A. Rupam Mahmood. Memory-efficient reinforce- ment learning with value-based knowledge consolidation. Transactions on Machine Learning Research,

    work page 2023

Show all 45 references
  1. [21]

    Pyhopper – Hyperparameter optimization.arXiv, 2210.04728,

    [Lechneret al., 2022 ] Mathias Lechner, Ramin Hasani, Philipp Neubauer, Sophie Neubauer, and Daniela Rus. Pyhopper – Hyperparameter optimization.arXiv, 2210.04728,

    2022 arXiv

  2. [22]

    How good is neural combinatorial optimization? A systematic evaluation on the traveling salesman problem

    [Liuet al., 2023 ] Shengcai Liu, Yu Zhang, Ke Tang, and Xin Yao. How good is neural combinatorial optimization? A systematic evaluation on the traveling salesman problem. IEEE Computational Intelligence Magazine, 18(3):14–28,

    2023

  3. [23]

    A learning-based iterative method for solving ve- hicle routing problems

    [Luet al., 2020 ] Hao Lu, Xingwen Zhang, and Shuang Yang. A learning-based iterative method for solving ve- hicle routing problems. InInternational Conference on Learning Representations,

    2020

  4. [24]

    Metabox: A benchmark platform for meta-black-box optimization with reinforcement learning

    [Maet al., 2023 ] Zeyuan Ma, Hongshu Guo, Jiacheng Chen, Zhenrui Li, Guojun Peng, Yue-Jiao Gong, Yining Ma, and Zhiguang Cao. Metabox: A benchmark platform for meta-black-box optimization with reinforcement learning. InAdvances in Neural Information Processing Systems, volume ...

    2023

  5. [25]

    Malan and Andries P

    [Malan and Engelbrecht, 2013] Katherine M. Malan and Andries P. Engelbrecht. A survey of techniques for char- acterising fitness landscapes and some possible ways for- ward.Information Sciences, 241:148–163,

    2013

  6. [27]

    Fuzzy tiling activations: A simple approach to learning sparse representations online

    [Panet al., 2021 ] Yangchen Pan, Kirby Banman, and Martha White. Fuzzy tiling activations: A simple approach to learning sparse representations online. InInternational Conference on Learning Representations,

    2021

  7. [28]

    Learn- ing from offline and online experiences: A hybrid adap- tive operator selection framework

    [Peiet al., 2024 ] Jiyuan Pei, Jialin Liu, and Yi Mei. Learn- ing from offline and online experiences: A hybrid adap- tive operator selection framework. InProceedings of the Genetic and Evolutionary Computation Conference, page 1017–1025, New York, NY , USA,

    2024

  8. [29]

    [Peiet al., 2025 ] Jiyuan Pei, Yi Mei, Jialin Liu, Mengjie Zhang, and Xin Yao

    ACM. [Peiet al., 2025 ] Jiyuan Pei, Yi Mei, Jialin Liu, Mengjie Zhang, and Xin Yao. Adaptive operator selection for meta- heuristics: A survey.IEEE Transactions on Artificial In- telligence, pages 1–21,

    2025

  9. [31]

    Deep reinforcement learning based parameter control in differ- ential evolution

    [Sharmaet al., 2019 ] Mudita Sharma, Alexandros Komni- nos, Manuel L ´opez-Ib´a˜nez, and Dimitar Kazakov. Deep reinforcement learning based parameter control in differ- ential evolution. InProceedings of the Genetic and Evo- lutionary Computation Conference, page 709–717, New ...

    2019

  10. [34]

    Springer US, Boston, MA,

    [Thrun, 1998] Sebastian Thrun.Lifelong Learning Algo- rithms, pages 181–209. Springer US, Boston, MA,

    1998

  11. [36]

    Meta- learning for black-box optimization

    [TVet al., 2020 ] Vishnu TV , Pankaj Malhotra, Jyoti Nar- wariya, Lovekesh Vig, and Gautam Shroff. Meta- learning for black-box optimization. InMachine Learning and Knowledge Discovery in Databases, pages 366–381. Springer International Publishing,

    2020

  12. [37]

    Incremental reinforcement learning in continuous spaces via policy relaxation and importance weighting

    [Wanget al., 2020 ] Zhi Wang, Han-Xiong Li, and Chunlin Chen. Incremental reinforcement learning in continuous spaces via policy relaxation and importance weighting. IEEE Transactions on Neural Networks and Learning Sys- tems, 31(6):1870–1883,

    2020

  13. [38]

    Recent advances in bayesian optimization.ACM Comput

    [Wanget al., 2023 ] Xilu Wang, Yaochu Jin, Sebastian Schmitt, and Markus Olhofer. Recent advances in bayesian optimization.ACM Comput. Surv., 55(13s), July

    2023

  14. [39]

    A comprehensive survey of continual learning: Theory, method and application.IEEE Trans- actions on Pattern Analysis and Machine Intelligence, 46(8):5362–5383,

    [Wanget al., 2024 ] Liyuan Wang, Xingxing Zhang, Hang Su, and Jun Zhu. A comprehensive survey of continual learning: Theory, method and application.IEEE Trans- actions on Pattern Analysis and Machine Intelligence, 46(8):5362–5383,

    2024

  15. [40]

    Graph as- sisted offline-online deep reinforcement learning for dy- namic workflow scheduling

    [Yanget al., 2025 ] Yifan Yang, Gang Chen, Hui Ma, Cong Zhang, Zhiguang Cao, and Mengjie Zhang. Graph as- sisted offline-online deep reinforcement learning for dy- namic workflow scheduling. InInternational Conference on Learning Representations,

    2025

  16. [41]

    Automated design of metaheuristics using rein- forcement learning within a novel general search frame- work.IEEE Transactions on Evolutionary Computation, 27(4):1072–1084,

    [Yiet al., 2023 ] Wenjie Yi, Rong Qu, Licheng Jiao, and Ben Niu. Automated design of metaheuristics using rein- forcement learning within a novel general search frame- work.IEEE Transactions on Evolutionary Computation, 27(4):1072–1084,

    2023

  17. [42]

    [Zhanget al., 2023 ] Tiantian Zhang, Xueqian Wang, Bin Liang, and Bo Yuan. Catastrophic interference in rein- forcement learning: A solution based on context division and knowledge distillation.IEEE Transactions on Neu- ral Networks and Learning Systems, 34(12):9925–9939,

    2023

  18. [43]

    Symbolic learning to optimize: Towards interpretability and scalability.arXiv, 2203.0657,

    [Zhenget al., 2022 ] Wenqing Zheng, Tianlong Chen, Ting- Kuei Hu, and Zhangyang Wang. Symbolic learning to optimize: Towards interpretability and scalability.arXiv, 2203.0657,

    2022

  19. [44]

    B Tree Structure Rule Representation We construct a solution updating rule (c.f., Section 3.2) as follows

    A State Representation The nine FLA metrics for state representation used in SYM- BOL [Chenet al., 2024 ] are also used in this work (c.f., Sec- tion 3.1), including (i) the average distance between any pair of solutions in the current population, (ii) the average dis- tance b...

    2024

  20. [100]

    Forall-taskmethod, 32 problems are sampled for each function, forming a set of 320 training problems in each epoch

    The learning rate is set to η= 0.001. Forall-taskmethod, 32 problems are sampled for each function, forming a set of 320 training problems in each epoch. The models follow the same setting as in SYM- BOL [Chenet al., 2024 ]. For the experiment for analysis of weights and ablat...

    2024

  21. [1983]

    Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, Demis Hass- abis, Claudia Clopath, Dharshan Kumaran, and Raia Had- sell

    [Kirkpatricket al., 2017 ] James Kirkpatrick, Razvan Pas- canu, Neil Rabinowitz, Joel Veness, Guillaume Des- jardins, Andrei A. Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, Demis Hass- abis, Claudia Clopath, Dharshan Kumaran, and Raia Had- sell. O...

    2017

  22. [1997]

    Learn to op- timize – A brief overview.National Science Review, 11(8):nwae132, 04

    [Tang and Yao, 2024] Ke Tang and Xin Yao. Learn to op- timize – A brief overview.National Science Review, 11(8):nwae132, 04

    2024

  23. [1998]

    Varga, Orly Avraham, Ziv Ben-Aharon, Alisa Khramushin, and Ora Schueler-Furman

    [Tsabanet al., 2022 ] Tomer Tsaban, Julia K. Varga, Orly Avraham, Ziv Ben-Aharon, Alisa Khramushin, and Ora Schueler-Furman. Harnessing protein folding neural net- works for peptide–protein docking.Nature Communica- tions, 13(1), January

    2022

  24. [2002]

    Blackbox optimization for approximating high- fidelity heat transfer calculations in metal additive manu- facturing.Results in Materials, 13:100258,

    [Biet al., 2022 ] Sirui Bi, Benjamin Stump, Jiaxin Zhang, Yousub Lee, John Coleman, Matt Bement, and Guannan Zhang. Blackbox optimization for approximating high- fidelity heat transfer calculations in metal additive manu- facturing.Results in Materials, 13:100258,

    2022

  25. [2012]

    Metaheuristic research: A comprehensive survey.Artificial Intelligence Review, 52(4):2191–2233,

    [Hussainet al., 2018 ] Kashif Hussain, Mohd Najib Mohd Salleh, Shi Cheng, and Yuhui Shi. Metaheuristic research: A comprehensive survey.Artificial Intelligence Review, 52(4):2191–2233,

    2018

  26. [2013]

    Problem definitions and evaluation criteria for the CEC 2021 on single objective bound con- strained numerical optimization

    [Mohamedet al., 2021 ] Ali Wagdy Mohamed, Anas A Hadi, Ali Khater Mohamed, Prachi Agrawal, Abhishek Kumar, and PN Suganthan. Problem definitions and evaluation criteria for the CEC 2021 on single objective bound con- strained numerical optimization. InIEEE Congress on Evolutio...

    2021

  27. [2014]

    Automated algorithm configu- ration and parameter tuning

    [Hoos, 2012] Holger H Hoos. Automated algorithm configu- ration and parameter tuning. InAutonomous search, pages 37–71. Springer,

    2012

  28. [2016]

    Evolution strategies – A comprehensive introduction.Natural Computing, 1(1):3–52,

    [Beyer and Schwefel, 2002] Hans-Georg Beyer and Hans- Paul Schwefel. Evolution strategies – A comprehensive introduction.Natural Computing, 1(1):3–52,

    2002

  29. [2017]

    Symbolic discovery of optimization algorithms

    [Chenet al., 2023 ] Xiangning Chen, Chen Liang, Da Huang, Esteban Real, Kaiyuan Wang, Hieu Pham, Xuanyi Dong, Thang Luong, Cho-Jui Hsieh, Yifeng Lu, and Quoc V Le. Symbolic discovery of optimization algorithms. In Advances in Neural Information Processing Systems, vol- ume 36,...

    2023

  30. [2018]

    Blackbox and derivative-free optimization: theory, algorithms and applications.Optimization and En- gineering, 17(1):1–2, February

    [Audet and Kokkolaras, 2016] Charles Audet and Michael Kokkolaras. Blackbox and derivative-free optimization: theory, algorithms and applications.Optimization and En- gineering, 17(1):1–2, February

    2016

  31. [2019]

    [Storn and Price, 1997] Rainer Storn and Kenneth Price

    Association for Computing Ma- chinery. [Storn and Price, 1997] Rainer Storn and Kenneth Price. Differential evolution – A simple and efficient heuristic for global optimization over continuous spaces.Journal of Global Optimization, 11(4):341–359,

    1997

  32. [2020]

    [Gomeset al., 2021 ] Hugo Siqueira Gomes, Benjamin L´eger, and Christian Gagn ´e

    International Foundation for Autonomous Agents and Multiagent Systems. [Gomeset al., 2021 ] Hugo Siqueira Gomes, Benjamin L´eger, and Christian Gagn ´e. Meta learning black-box population-based optimizers,

    2021

  33. [2021]

    Meta- learning of black-box solvers using deep reinforcement learning

    [Chayboutiet al., 2022 ] Sofian Chaybouti, Ludovic Dos Santos, Cedric Malherbe, and Aladin Virmaux. Meta- learning of black-box solvers using deep reinforcement learning. InSixth Workshop on Meta-Learning at the Con- ference on Neural Information Processing Systems,

    2022

  34. [2022]

    Improving differential evolution through Bayesian hyperparameter optimization

    [Biswaset al., 2021 ] Subhodip Biswas, Debanjan Saha, Shuvodeep De, Adam D Cobb, Swagatam Das, and Brian A Jalaian. Improving differential evolution through Bayesian hyperparameter optimization. InIEEE Congress on Evolutionary Computation, pages 832–840,

    2021

  35. [2023]

    SYM- BOL: Generating flexible black-box optimizers through symbolic equation learning

    [Chenet al., 2024 ] Jiacheng Chen, Zeyuan Ma, Hongshu Guo, Yining Ma, Jie Zhang, and Yue-Jiao Gong. SYM- BOL: Generating flexible black-box optimizers through symbolic equation learning. InInternational Conference on Learning Representations,

    2024

  36. [2024]

    Improving performance in reinforcement learning by breaking generalization in neu- ral networks

    [Ghiassianet al., 2020 ] Sina Ghiassian, Banafsheh Rafiee, Yat Long Lo, and Adam White. Improving performance in reinforcement learning by breaking generalization in neu- ral networks. InInternational Conference on Autonomous Agents and MultiAgent Systems, page 438–446, Rich- ...

    2020

  37. [2025]

    Proximal policy optimization algorithms.arXiv, 1707.06347,

    [Schulmanet al., 2017 ] John Schulman, Filip Wolski, Pra- fulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms.arXiv, 1707.06347,

    2017 arXiv

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.