{"id":"ca924f4f-dacd-4366-b66b-14e76cfcc868","arxiv_id":"2505.13025","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"LiBOG combines inter-task elastic weight consolidation and a new intra-task elite behavior consolidation, enabling a reinforcement-learning-based black-box optimizer generator to learn sequential task distributions with less forgetting than fine-tuning.","lead":"LiBOG is a training method that lets an AI-generated black-box optimizer keep learning from new problem types as they arrive, instead of freezing after a single training phase. It combines elastic weight consolidation with a new elite-behavior penalty and shows reduced catastrophic forgetting on sequential benchmark optimization tasks.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Reported experimental advantages rest on normalized score differences smaller than the run-to-run noise, with no significance testing; the 'eight of twelve' ranking may not be a real effect.","rationale":"The reader's stated weakest assumption was the known-task-boundary requirement. That is a genuine limitation, but it is explicitly scoped in Section 3.1 and in the conclusion, so it does not threaten the central claim within the paper's chosen setting. The more load-bearing concern for the in-scope claim is the adequacy of the empirical evidence: the paper uses mean normalized scores with very small absolute differences, reports no statistical tests, and bases its 'first in eight of twelve' conclusion on rank ordering over those means. Since the central claim is comparative and empirical, the weakest link is whether the reported differences survive a simple noise check. This is testable directly from the released code and per-run data. If the differences do survive, the paper's central claim is solid and the existing conditional verdict can be lifted to acceptance; if they do not, the claim that LiBOG 'significantly mitigates catastrophic forgetting' and maintains plasticity is only weakly supported. The verdict should therefore remain conditional pending this check, which matches the reader's conditional assessment but highlights a different primary reason.","tokens_in":17998,"tokens_out":8848,"duration_ms":99741,"concrete_test":"Extract the per-run results from the released repository for each method, task, and task order. For each task case, compute the difference LiBOG minus fine-tuning and LiBOG minus restart, using either the normalized scores or the raw final objective values. If the 10 runs per method are seed-paired, use a paired bootstrap with 10,000 resamples; otherwise use a two-sample bootstrap or permutation test. Report 95% confidence intervals for each difference and apply a Holm-Bonferroni correction across the 12 task cases. Also run the same procedure on the five-run ablation results in Table 2, contrasting LiBOG against only-intra and only-inter. If fewer than about half of the corrected intervals exclude zero, the claim that LiBOG's advantage is 'significant' is unsupported; if most intervals exclude zero, the concern is resolved and the central claim is strengthened.","verdict_should_be":"UNCHANGED","load_bearing_attack":"LiBOG's central claim is that its two consolidation terms deliver mild catastrophic forgetting and good plasticity relative to fine-tuning and restart. The main evidence is Table 1's rank summary, but that summary is computed from mean normalized scores whose differences are often comparable to or smaller than the reported standard deviations. For example, in order 0, LiBOG versus fine-tuning differs by 0.000389 on P0 and 0.000675 on P1, while the reported standard deviations are on the order of 0.0005-0.0011 and 0.0046-0.0087 over n=10 runs. No p-values, confidence intervals, or effect sizes are reported anywhere. The ablation comparison is more striking: LiBOG (0.982440) versus only-intra (0.982306) differs by 0.000134 over five runs, yet the text states that removing either component 'significantly reduces' performance; only the only-inter drop looks larger than noise. Because the ranks in Table 1 are derived from these small mean differences, the headline result that LiBOG is first in eight of twelve cases could plausibly be driven by run-to-run noise rather than by a systematic advantage. The paper's own normalization (Appendix D.2) compresses all scores toward 1, which makes the absence of inferential statistics especially consequential. The qualitative forgetting curves in Figure 3 do suggest a real effect for LiBOG relative to fine-tuning on early tasks, so the concern is not that the method is useless; it is that the quantitative comparative claims are not established at the reported precision.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces LiBOG, a lifelong learning framework for meta-black-box optimization (MetaBBO). The setting is a sequence of tasks, each defined by a problem distribution; the goal is a single model that generates symbolic solution-update rules (optimizers) that work well on all previously seen distributions. LiBOG builds on SYMBOL's PPO-based symbolic equation learning and adds two consolidation mechanisms: inter-task elastic weight consolidation (EWC; Eq. 3) and an intra-task elite behavior consolidation (EBC; Eq. 4) that regularizes the current policy toward an elite policy by KL divergence. Experiments on three task orders constructed from CEC benchmark function categories compare LiBOG against restart, fine-tuning, all-task training, and MadDE, reporting ranks, forgetting curves, sensitivity analysis, and an ablation study. The paper claims that LiBOG mitigates catastrophic forgetting while retaining plasticity, is ranked first in eight of twelve task cases, and is robust across task orders and hyper-parameter settings.","tokens_in":18288,"tokens_out":6011,"duration_ms":57477,"significance":"If the claims hold, this is one of the first lifelong learning approaches for MetaBBO, addressing a practically important limitation of current one-off MetaBBO training. The idea of consolidating both inter-task and intra-task knowledge is sensible, and the qualitative forgetting curves (Figure 3) suggest that the proposed consolidation terms do reduce catastrophic forgetting relative to fine-tuning. The paper also ships code, and the ablation and sensitivity studies give useful information about the method's behavior. However, the quantitative evidence for the headline claims is currently weak: the normalized score differences between LiBOG and baselines are often smaller than the reported run-to-run standard deviations, and no significance testing is provided. In addition, the mathematical definitions of the consolidation terms contain inconsistencies and unspecified computational details that are central to the method. These issues are fixable but require substantive revisions.","major_comments":[{"comment":"The storage complexity claim for inter-task consolidation is inconsistent with the equations. Eq. (2) defines Ω_i as an outer product ∇ℓ∇ℓ^T, which is a full d×d matrix, and Eq. (3) uses Ω_j inside a quadratic form (θ−θ*_j)^T Ω_j (θ−θ*_j). This would require O(|θ|^2) storage per task and O(|θ|^2 I) overall, contradicting the stated O(|θ| I) space complexity. If the implementation actually uses a diagonal approximation (as is standard in EWC), the authors must state this explicitly and define Ω_i as the diagonal of the outer-product matrix (or as the average squared gradient), and they must clarify how Eq. (3) is computed under that approximation.","section":"Section 3.3, Eqs. (2)-(3)"},{"comment":"Equation (4) defines the intra-task consolidation loss as a sum over all actions a∈A of π_θ(a|s) log(π_θ(a|s)/π_θe(a|s)). In this paper, an action is a complete tree-structured symbolic update rule generated autoregressively by an LSTM, so the action space is exponentially large (or effectively unbounded). The full sum over A is therefore intractable as written. The paper must specify how CL_intra is actually computed in practice: for example, whether it uses a per-step categorical KL divergence over the node vocabulary, a Monte Carlo estimate from the sampled trajectories, or a closed-form expression exploiting the specific policy parametrization. Without this detail the method is not fully specified and cannot be reproduced.","section":"Section 3.4, Eq. (4)"},{"comment":"The headline ranking claim that LiBOG is first in eight of twelve cases is based on mean normalized scores whose differences are often smaller than the reported standard deviations. For instance, in Table 4 (order 0), LiBOG on P0 is 0.999138±0.000499 vs fine-tuning 0.998749±0.001087 (difference 0.000389), and on P1 is 0.925060±0.004585 vs 0.924385±0.008710 (difference 0.000675), both well within one standard deviation of each other. No p-values, confidence intervals, or effect sizes are reported anywhere. Similarly, in the ablation (Table 2), the difference between LiBOG (0.982440) and only-intra (0.982306) is 0.000134 over five runs, yet the text states that removing either component 'significantly reduces' performance. The authors should add appropriate inferential statistics (e.g., paired tests across the 10 runs, or confidence intervals on the ranks) or substantially soften the quantitative comparative claims. The forgetting curves in Figure 3 provide qualitative support for the method's benefit on early tasks, but they do not justify the 'eight of twelve' statement.","section":"Section 4.1 and Tables 4-6"}],"minor_comments":[{"comment":"The layout of Table 1 is garbled and very hard to interpret; the column grouping under each task order is unclear, and the rank entries for restart and all-task appear to overlap across orders. Please reformat the table so each task order is clearly separated and the values are unambiguous.","section":"Table 1"},{"comment":"There is a typo: 'the potion of remaining optimization iterations' should be 'the portion of remaining optimization iterations'.","section":"Appendix A"},{"comment":"The baseline name 'fintuning' should be 'fine-tuning'.","section":"Table 5"},{"comment":"The sentence in the ablation study, 'removing either significantly reduces its performance,' is not supported by the reported numbers or any statistical test; please rephrase to reflect the magnitude of the observed differences or add a test.","section":"Section 4.3"},{"comment":"The text says 'the best-performance weight values above are used' for the ablation, but the LiBOG row in Table 8 (e.g., P0=0.999528, P3=0.994326) does not match any row in Table 7 (e.g., α=10,β=1 gives P0=0.999714, P3=0.993138). Please clarify which weight setting was used and why the numbers differ.","section":"Section 4.3 and Table 8"},{"comment":"The statement that 'LiBOG outperforms all baseline methods for each of the three task orders' is too strong given that restart achieves the best performance in some individual cases (as acknowledged later). Please rephrase to indicate that LiBOG has the best average rank per order, or otherwise clarify the comparison.","section":"Section 4.1"}],"recommendation":"major_revision","confidential_remarks":"The paper presents a promising first step toward lifelong MetaBBO, and the code release is a strength. The main concerns are (i) the inconsistency between the EWC equations and the claimed storage complexity, (ii) the unspecified and potentially intractable computation of the KL term in Eq. (4), and (iii) the absence of significance testing for the quantitative comparative claims. All three are fixable with careful revision, so I recommend major revision rather than rejection. I would also suggest the authors double-check the action-space formulation of Eq. (4), since if the KL sum is truly over all complete trees, the method as written cannot be implemented exactly."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is the first MetaBBO paper I've seen that treats distribution shift as a lifelong RL problem, and the elite behavior consolidation idea is genuinely new. The forgetting curves in Figure 3 show a qualitative effect: fine-tuning collapses on earlier tasks, LiBOG doesn't. That part is convincing. The paper also ships code, which makes the ambiguities fixable.\n\nThe main claim I'd push back on is the 'eight of twelve' ranking. The ranks come from mean normalized scores, and in several cases the LiBOG-versus-fine-tuning gap is smaller than the reported standard deviation over ten runs. Order 0 P0: 0.000389 difference with SDs around 0.0005-0.0011. Order 0 P1: 0.000675 with SDs 0.0046-0.0087. The ablation gap between LiBOG and only-intra is 0.000134 over five runs. No p-values, CIs, or effect sizes anywhere. The text says removing either component 'significantly reduces' performance, but only the only-inter drop looks outside noise. So Table 1 should be read as suggestive, not established. This is the load-bearing quantitative evidence for the comparative claim, and it needs a statistical pass before the paper is used as a benchmark.\n\nTwo smaller issues: Eq (2) writes a full outer-product importance matrix while Eq (3) and the stated O(|theta| I) storage imply a diagonal approximation; the paper never says which is implemented. And the known-task-boundary assumption is a real restriction, though the authors state it plainly in the conclusion, so it's an honest limitation rather than a hidden one. The CEC-only benchmark is narrow but fine for a first step.\n\nI'd send it to peer review. The paradigm is new, the mechanism is concrete, the code is there, and the forgetting curves give real signal. But I'd ask for significance testing, a clearer EWC implementation statement, and a revised claim about the rank results. Who benefits: MetaBBO researchers and anyone doing continual RL for optimization. I'd cite it if I worked in either area.","headline":"A useful first step for lifelong MetaBBO with a plausible consolidation recipe, but the quantitative ranking claims are not backed by significance testing.","tokens_in":18814,"tokens_out":1759,"would_cite":true,"duration_ms":19505,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"LiBOG claims that lifelong reinforcement learning can generate black-box optimizers for a sequence of arriving problem distributions, keeping performance on earlier distributions nearly intact while still learning new ones.","keywords":["lifelong learning","continual learning","meta black-box optimization","black-box optimizer generation","catastrophic forgetting","symbolic equation learning","reinforcement learning","elastic weight consolidation"],"falsifier":"A replication on the same three task orders would falsify the central claim if, after training the full sequence, LiBOG's normalized performance on an early task fell below that of fine-tuning on the same task, or if removing either consolidation term left final performance unchanged. A sharper check is to run the same training protocol with hidden task boundaries or gradual changes in the benchmark's offset and rotation parameters; if forgetting returns and LiBOG no longer beats fine-tuning, the method's applicability is limited to the clean-block regime it assumes.","tokens_in":17797,"feed_emoji":"🧠","tokens_out":8183,"duration_ms":81235,"temperature":0.7,"pith_summary":"Meta-black-box optimization has so far trained a model once on a fixed distribution of problems and then frozen it, but real optimization workloads arrive as a stream of related, shifting distributions. LiBOG treats each arriving distribution as a task in a non-stationary Markov decision process and keeps training one policy that emits symbolic solution-update rules, so a single model serves every distribution it has seen. It combines two consolidation mechanisms—elastic weight consolidation across tasks and a new elite-behavior consolidation within tasks—to fight catastrophic forgetting while preserving plasticity. On four benchmark problem categories arranged in three different task orders, the paper reports that LiBOG ranks first in eight of twelve task-position cases and that removing either consolidation term significantly hurts performance. The payoff, if true, is that automated optimizer design can keep improving as new problem distributions appear instead of being retrained from scratch or frozen after one training run.","feed_headline":"Lifelong trainer keeps old optimizers sharp while learning new tasks","feed_subtitle":"LiBOG adds two consolidation terms so a stream of problem distributions rarely erases earlier know-how.","key_machinery":"The load-bearing object is the composite loss $L = L_{\\mathrm{PPO}} + \\alpha\\,CL_{\\mathrm{inter}} + \\beta\\,CL_{\\mathrm{intra}}$, where $CL_{\\mathrm{inter}}$ is the EWC penalty built from per-task importance matrices $\\Omega_j$ and stored task parameters $\\theta_j^*$, and $CL_{\\mathrm{intra}}$ is the KL penalty that pulls the current policy toward an elite policy $\\pi_{\\theta_e}$ on observed states. The surrounding machinery is the non-stationary MDP formulation that turns each arriving problem distribution into a task, the LSTM that generates symbolic update rules, and the elite-update rule that copies the current model whenever its average final objective beats the stored elite. This combination lets the model preserve old behaviors without storing raw experience.","core_discovery":"The central discovery is that catastrophic forgetting in meta-black-box optimization can be controlled by pairing parameter-space and behavior-space consolidation in a single lifelong reinforcement-learning objective. LiBOG models lifelong learning as a non-stationary MDP built from stationary MDPs, one per problem distribution, with a shared state space of fitness-landscape metrics and a shared action space of symbolic update trees. An LSTM policy trained with PPO produces the tree-structured rules for each optimization step. After each task, the method stores the task's parameters and a per-parameter importance matrix and adds the EWC penalty $CL_{\\mathrm{inter}}$ (Eq. 3) to all later losses; during a task, it maintains an elite policy—the best-performing model seen so far by average final objective—and penalizes the KL divergence between current and elite action distributions via $CL_{\\mathrm{intra}}$ (Eq. 4). The paper demonstrates on the CEC uni-modal, basic, hybrid, and composition categories that with both terms the test performance on earlier tasks stays almost flat after later training, that new tasks are still learned well, and that both consolidation mechanisms contribute to the final performance.","pith_inferences":["The same EWC-plus-elite-behavior recipe does not depend on symbolic rules: it could be grafted onto other MetaBBO designs, such as end-to-end solution generators or parameter-configuration policies, because it acts on any policy's parameters and action distributions.","If task boundaries were unknown or distributions drifted smoothly, LiBOG's stored-task bookkeeping would need to be replaced by online change-point detection or a generative model of tasks; the known-boundary assumption marks that as the next regime to test.","Because the reported normalized scores are compressed (the supplement's raw-objective example shows LiBOG at 2945.91 versus 4354.64 on the final task of one order), small normalized differences may correspond to large practical gaps; re-reporting on raw objective values would make the comparison easier to interpret.","A robustness extension would replace the elite-update criterion of average final objective with a robust statistic such as median or worst-case performance, since black-box runs are noisy; whether EBC's benefit survives noisier reward estimates is not quantified in the paper."],"forward_implications":["A single optimizer-generation model can be deployed across a growing set of problem distributions, so new arrivals do not force a freeze-and-discard retraining cycle.","Because only parameters and importance matrices are stored per task, memory overhead stays low ($O(|\\theta|\\cdot I)$) rather than requiring large experience buffers.","Starting each new task from transferred knowledge gives LiBOG usable performance on a new distribution before it has been trained on it, an effect the paper documents at task boundaries.","Both consolidation terms are necessary: the reported ablation shows that removing either one lowers average normalized performance, with the intra-task term having the larger effect in that comparison."],"supporting_citations":[{"why":"Provides the symbolic equation learning LSTM policy, PPO training, state representation, and reward function on which LiBOG's rule construction is built.","marker":"[Chen et al., 2024]"},{"why":"Supplies elastic weight consolidation, the inter-task parameter-regularization mechanism with per-parameter importance.","marker":"[Kirkpatrick et al., 2017]"},{"why":"PPO is the reinforcement-learning algorithm whose loss forms the base term of LiBOG's combined objective.","marker":"[Schulman et al., 2017]"},{"why":"Defines the CEC benchmark problem categories used to construct the sequential tasks.","marker":"[Mohamed et al., 2021]"},{"why":"MadDE is the human-designed optimizer used both as a baseline and as the guide optimizer in the reward calculation.","marker":"[Biswas et al., 2021]"},{"why":"Supplies the fitness landscape analysis metrics used in the shared state representation.","marker":"[Malan and Engelbrecht, 2013]"},{"why":"Motivates intra-task catastrophic forgetting in reinforcement learning, the phenomenon EBC targets.","marker":"[Igl et al., 2021]"}],"fun_headline_variants":["Learn optimizers forever without forgetting old tricks","MetaBBO goes lifelong: stop retraining from scratch","Lifelong optimizer evolution with dual consolidation","Keep old optimizers, learn new ones: LiBOG does both","No more catastrophic forgetting in black-box optimization"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"LiBOG assumes the identity and arrival boundary of each problem distribution is known, so every task is a clean stationary block and can be stored as a separate parameter-plus-importance record; if distributions drift continuously or switch without labels, the method cannot be applied as described.","fun_headline_variants_meta":{"raw":{"variants":["Learn optimizers forever without forgetting old tricks","MetaBBO goes lifelong: stop retraining from scratch","Lifelong optimizer evolution with dual consolidation","Keep old optimizers, learn new ones: LiBOG does both","No more catastrophic forgetting in black-box optimization"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000246,"raw_usage":{"total_tokens":1557,"prompt_tokens":977,"completion_tokens":580,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":593,"completion_tokens_details":{"reasoning_tokens":505}},"tokens_in":593,"tokens_out":580,"duration_ms":6528,"temperature":1.0,"reasoning_tokens":505,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T20:21:38.520986+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A replication on the same three task orders would falsify the central claim if, after training the full sequence, LiBOG's normalized performance on an early task fell below that of fine-tuning on the same task, or if removing either consolidation term left final performance unchanged. A sharper check is to run the same training protocol with hidden task boundaries or gradual changes in the benchmark's offset and rotation parameters; if forgetting returns and LiBOG no longer beats fine-tuning, the method's applicability is limited to the clean-block regime it assumes.","supporting_citations":[{"cited_title":"Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, Demis Hass- abis, Claudia Clopath, Dharshan Kumaran, and Raia Had- sell","cited_arxiv_id":null,"evidence_quote":"Supplies elastic weight consolidation, the inter-task parameter-regularization mechanism with per-parameter importance."},{"cited_title":"Problem definitions and evaluation criteria for the CEC 2021 on single objective bound con- strained numerical optimization","cited_arxiv_id":null,"evidence_quote":"Defines the CEC benchmark problem categories used to construct the sequential tasks."},{"cited_title":"Improving differential evolution through Bayesian hyperparameter optimization","cited_arxiv_id":null,"evidence_quote":"MadDE is the human-designed optimizer used both as a baseline and as the guide optimizer in the reward calculation."},{"cited_title":"Malan and Andries P","cited_arxiv_id":null,"evidence_quote":"Supplies the fitness landscape analysis metrics used in the shared state representation."},{"cited_title":"Transient non-stationarity and generalisation in deep rein- forcement learning","cited_arxiv_id":null,"evidence_quote":"Motivates intra-task catastrophic forgetting in reinforcement learning, the phenomenon EBC targets."}],"review_version":1}