Pith. sign in

REVIEW 2 major objections 6 minor 38 references

Cooperative Parameter-subspace Evolution Strategy (CoPES) claims that splitting the parameter search into lower-dimensional subspaces makes evolution-strategy post-training of tool-using agents competitive with gradient-based RL under fixed

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-04 08:08 UTC pith:E43MWRC5

load-bearing objection CoPES is a promising new method for ES-based LLM post-training, but the paper's central claim that subspace decomposition drives the gains is undercut by an uncontrolled step-size increase and single-seed runs. the 2 major comments →

arxiv 2608.02391 v1 pith:E43MWRC5 submitted 2026-08-03 cs.AI cs.LG

Cooperative Coevolution for Resource-Constrained Agentic LLM Post-Training

Zhiyuan Wang , Shengcai Liu , Jiahao Wu , Ning Lu , Hui Ouyang , Shaofeng Zhang , Haoze Lv , Ke Tang This is my paper
classification cs.AI cs.LG
keywords cooperative coevolutionevolution strategiesLLM post-trainingtool-using agentsmemory-efficient RLGRPOmath reasoningmulti-hop QA
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Tool-using LLM agents generate long multi-turn trajectories, so gradient-based post-training is memory-bound. The paper's central proposal is to make evolution strategies—which only need forward passes and rewards—efficient enough for this setting by applying cooperative coevolution: randomly partition all model parameters into K equally sized subspaces at each step, perturb one subspace at a time under the shared full-model context, standardize rewards jointly across subspaces, and compose the subspace updates. On a 4B-parameter math agent under the GPU-hour budget of the best full-parameter GRPO validation checkpoint, CoPES recovers 92% of GRPO's validation-accuracy gain versus 67% for standard ES, and it beats standard ES and LoRA-based GRPO on all five math benchmarks on pass@k; the same ordering appears on a multi-hop QA task. The paper is transparent that the memory advantage is a theoretical accounting (about one-eighth of full-parameter GRPO at 128K context) and that each training configuration was run once, so the single-run estimate should be read with that caveat.

Core claim

On the paper's own terms, the discovery is that CoPES reallocates a fixed evaluation budget from d-dimensional perturbations to d/K-dimensional perturbations without reducing the expected perturbation norm (each subspace noise is scaled by √K), and that this improves the quality of the gradient-free update estimate. Because each scalar reward now reflects a lower-dimensional change, the same N trajectory evaluations produce more informative directions. CoPES composes these estimates into one synchronous full-parameter update, so it remains full-parameter and forward-only. The paper's experiments claim this yields fixed-budget gains on five math benchmarks and three QA benchmarks, and that th

What carries the argument

The driving mechanism is the cooperative subspace update: at every step, sample a random equal-size partition of parameter indices, allocate N/K perturbations per subspace, scale noise by σ_k = √K σ to match the expected squared norm of full-space noise, pool and jointly z-score all rewards, and sum the K subspace direction estimates into one synchronous full-parameter update. The random re-partitioning and the shared full-model context are what make the subspace estimates mutually comparable; joint standardization is the component that puts them on a common scale. Memory efficiency comes from seed replay, chunked parameter processing, and a CPU-side weight backup, so no perturbation vectors

Load-bearing premise

The load-bearing premise is that each method's single training run (stated in Supplementary A.5) captures ordering rather than seed noise; the paper's own K=8 ablation moves GSM8K pass@1 from 92.57 to 37.32 under otherwise identical settings, and AIME 2024/2025 each contain only 30 problems, where a ~7-point gap is about two correct answers.

What would settle it

Re-run CoPES, standard ES, and LoRA-based GRPO with at least 10 seeds under the identical 16-step budget, and compare pass@1 on AIME 2024/2025 and validation accuracy at the selected checkpoint. If the CoPES-over-standard-ES gap is within one or two correct answers across seeds, the central 92%-versus-67% recovery claim is not quantitatively supported. A second falsifier: if K=8 with N_k=5 performs close to K=4 when repeated across seeds, then the per-subspace population explanation is wrong.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • If CoPES is correct, backpropagation-free full-parameter post-training can operate within the same GPU-hour budget as gradient-based RL, removing a central argument for LoRA or for needing large GPU clusters.
  • A single 24GB GPU at 128K context becomes a feasible training configuration for agentic post-training, since memory scales with model weights plus KV cache rather than with activation depth.
  • CoPES's gain is not task-specific to math rewards: it also beats standard ES on all nine QA benchmark–metric pairs, and exceeds full-parameter GRPO at k≥2 on two QA datasets.
  • The K=4 setting with N/K≥10 becomes a practical recipe; the K=8 collapse shows the method's performance depends on keeping per-subspace population size above about 10.
  • CoPES retains the forward-only, full-parameter nature of ES, so it inherits ES's hardware-feasibility advantages without giving up the ability to update all model parameters.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: the random re-partition each step means every parameter is eventually co-optimized with different partners; a direct comparison of resampled versus fixed groupings would show whether this temporal mixing, rather than low dimensionality per se, drives the gain.
  • The method's failure at K=8 suggests adaptive partition size—coarser groups where parameters interact, finer groups where they are independent—could improve on uniform random partitioning; the paper itself lists adaptive partitioning as future work.
  • Because single runs underpin the headline result, the 92%-versus-67% figure is a directional estimate; repeated-seed replication is the natural next experiment before treating the magnitude as settled.
  • The same decomposition idea may transfer to other forward-only optimizers beyond ES, such as zeroth-order or evolutionary updates, whenever trajectory evaluation is expensive and the reward is scalar.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 6 minor

Summary. The paper proposes CoPES, a cooperative-coevolution variant of evolution strategies for post-training tool-using LLM agents under limited GPU resources. CoPES randomly partitions the full parameter space into K equal subspaces, allocates N/K perturbations per subspace, scales subspace perturbations by sqrt(K)*sigma, jointly standardizes rewards across subspaces, and composes the K subspace updates into a single full-parameter update. The authors post-train Qwen3.5-4B on the MATH task and HotpotQA-based multi-hop QA, comparing against full-parameter GRPO, LoRA-based GRPO, and standard ES. They report that, under a fixed 16-step budget set by full-parameter GRPO's best validation checkpoint, CoPES outperforms standard ES and LoRA-GRPO on pass@k across five math benchmarks and recovers 92% of GRPO's validation-accuracy gain versus 67% for standard ES, while retaining single-GPU feasibility and low memory. Ablations vary K, compare joint versus independent reward standardization, and include an ES-2sigma control.

Significance. If the central claim holds, CoPES is a practically meaningful contribution: a backpropagation-free, full-parameter post-training method that lowers both GPU memory and wall-clock training time relative to standard ES while approaching GRPO-level performance. The derivation of the subspace perturbation scale in Eqs. (10)-(11) is clean, the memory accounting is transparent, and the code is open-sourced. The ablations target the right alternatives. However, the current evidence is single-run and the main attribution test does not control for a change in effective update magnitude, so the quantitative headline is not yet established to the standard required for a strong claim.

major comments (2)
  1. [Eq. (13) and Table 3 (Perturbation-Scale Control)] The ES-2sigma ablation does not control for the increase in effective update scale. For standard ES (Eq. 6), each coordinate of the update has variance proportional to alpha^2/N. For CoPES, a coordinate in subspace k receives the sum of N_k = N/K terms of the form (alpha/N_k)*r_hat*epsilon, giving per-coordinate update variance proportional to alpha^2*K/N. Thus K=4 doubles the expected update norm relative to standard ES. The ES-2sigma ablation doubles sigma only; because rewards are standardized, it does not change the update formula and cannot detect this confound. The K-sweep is also consistent with a step-size effect: performance improves as K increases from 1 to 4 as the update norm grows, and collapses at K=8 when per-subspace sample size falls to N_k=5. To attribute the gains to cooperative subspace decomposition, the authors should compare CoPES with alpha divided by sqrt(K) (or
  2. [Supplementary A.5: 'Each reported training configuration is run once'] The headline comparisons (e.g., '92% vs 67% recovery', 'consistently outperforms standard ES') rest on one run per configuration. On AIME 2024 and AIME 2025, which contain only 30 problems each, the reported ~7-point pass@1 differences correspond to about two additional correct answers. The K-ablation itself shows large sensitivity across configurations on GSM8K (92.57 at K=4 versus 37.32 at K=8), which indicates that the pipeline can produce large outcome differences. At minimum, the main comparison and the ablations should be repeated with several seeds and reported with means and standard errors, or the quantitative claims should be appropriately tempered. This is load-bearing because the abstract reports specific recovery percentages.
minor comments (6)
  1. [Table 1 and abstract] The abstract says 'Under the GPU-hour budget of full-parameter GRPO's best validation checkpoint', but the 16-step block is matched by update count and unique prompt batches, with estimated GPU-hours differing (78.61 for GRPO variants, 68.65 for ES variants). The footnote discloses this, but the wording overstates the precision. Suggest 'within the GPU-hour budget' plus exact hours, or an exact GPU-hour-matched comparison.
  2. [Eq. (13) and Fig. 1] The text calls the update 'asynchronous' at the algorithmic level, but all g_k are computed from the same pre-update model and applied together. 'Synchronous composition' would be clearer and consistent with the Introduction.
  3. [Supp. B, Eq. (21)] The theoretical memory accounting uses a context-dependent term of 3 GB per K tokens for both GRPO variants. This is a lower-bound estimate; please provide a derivation or cite the dominant tensors more explicitly so that the 30x/35x memory-advantage statements can be checked.
  4. [Table 1 and Supp. A.1] MATH-500 is a subset of MATH-Test. This is disclosed, but reporting both as separate benchmarks in the main tables makes the evidence look broader than it is. Consider noting this directly in the main text or consolidating the presentation.
  5. [Eq. (11) and Algorithm 1] The derivation assumes d and N are divisible by K. In practice this may not hold for arbitrary checkpoints; state how a non-divisible d is handled (e.g., unequal final subspace) in the implementation notes.
  6. [Figure S4] The caption describes the CoPES dashed line as marking 'validation accuracy at its 16-step fixed-budget endpoint rather than a training curve.' This is useful, but the figure itself could label the dashed line directly to avoid confusion.

Circularity Check

0 steps flagged

No significant circularity: CoPES's update equations are derived from norm-matching and its headline gains are measured against external baselines.

full rationale

The derivation chain is self-contained. CoPES's three design choices are explicit algorithmic steps: the subspace partition (Eq. 7), the dimension-aware perturbation scale sigma_k = sqrt(K) sigma obtained by matching expected squared norms E||delta||^2 = d sigma^2 and E||delta_k||^2 = d_k sigma_k^2 (Eqs. 10-11), and joint reward standardization (Eq. 12) followed by the composed update (Eq. 13). None of these equations is fitted to the benchmarks or defined in terms of the reported pass@k or validation-accuracy outcomes. The central quantitative claims are comparisons against external baselines: full-parameter GRPO, LoRA-based GRPO, and standard ES under the same 16-step budget; the 92%-vs-67% recovery figure is computed from validation accuracies (49.20 baseline, 91.47 GRPO, 88.00 CoPES, 77.60 standard ES at the matched budget), not from fitted parameters. Note that the 67% figure refers to standard ES's same-budget row, not its later validation-selected checkpoint. K=4 is stated to have been fixed before test evaluation. The paper cites its own prior work (Safe Delta, AHDAgent, Train at Moving Edge) only as background on LLM safety, agentic RL, and prompt selection; none of these citations carries the derivation of CoPES or the uniqueness of its design. The remaining concerns about the results are statistical and experimental (single-run configurations, 30-problem AIME sets, and the uncontrolled effective update-scale confound raised by a skeptical reader) rather than circularity, since no prediction reduces by construction to an input.

Axiom & Free-Parameter Ledger

2 free parameters · 4 axioms · 0 invented entities

Everything the central claim rests on that is not derived in-paper: the classical cooperative-coevolution assumption that partially-perturbed models yield usable reward signals (shown fragile at K=8), comparability of rewards across subspaces for joint standardization, input-noise orthogonality of the composed update, and the stated memory-accounting model. The only hand-set free parameter proper to the method is K; the ES hyperparameters are inherited from prior work. No new entities are introduced.

free parameters (2)
  • K (subspace count) = 4
    Hand-chosen for both tasks before test evaluation (Supp. A.5). Performance is highly sensitive: GSM8K pass@1 drops from 92.57 (K=4) to 37.32 (K=8) (Table 3), so the headline result sits on a cliff controlled by this integer.
  • ES hyperparameters (N, sigma, alpha) = N=40, sigma=1e-3, alpha=5e-4
    Taken unchanged from cited ES-LLM work (Qiu et al. 2026; Liu et al. 2025), not tuned in this paper; they set the perturbation geometry that CoPES modifies via sqrt(K). Listed for completeness.
axioms (4)
  • domain assumption Rewards from single-subspace perturbations (all other weights fixed) carry enough signal about that subspace's update direction to improve on full-space perturbations under a fixed budget.
    Invoked throughout Cooperative Subspace Search (Methodology, Eqs. 8-9) as motivation ('each scalar reward reflects a lower-dimensional change... allowing the fixed evaluation budget to be used more effectively'). The K=8 collapse (Table 3) shows the assumption holds only in a regime window.
  • domain assumption Subspace rewards share a common scale, so pooling all N rewards into one mean/std (joint standardization) is valid.
    Methodology, Joint Reward Standardization (Eq. 12). The Indep. Z-score ablation (Table 3) supports this empirically at K=4, but the comparability of reward distributions across arbitrary random subspaces is not proven.
  • standard math Disjoint parameter supports give mutually orthogonal subspace estimates that compose into a full-space update.
    Methodology, Cooperative Subspace Search ('Because the subspaces have disjoint supports...'). Orthogonality holds for the input noise; reward noise is shared across subspaces through the common batch, so the estimates are not statistically independent.
  • domain assumption The theoretical GPU-memory model (FlashAttention, KV caching; 3 GB/K-token GRPO slope, 0.032 GB/K-token ES slope) is an accurate accounting of algorithm-level memory.
    Supplementary B, Eqs. S19-S21. A stated accounting model with claimed conservative rounding; it is an estimate of algorithm-level memory, not measured peak GPU memory, and the hardware-feasibility table is the only empirical check.

pith-pipeline@v1.3.0-daily-deepseek · 17994 in / 22239 out tokens · 223593 ms · 2026-08-04T08:08:46.168115+00:00 · methodology

0 comments
Cite this review

Pith. "Pith review of Cooperative Coevolution for Resource-Constrained Agentic LLM Post-Training." pith.science (2026). https://pith.science/paper/E43MWRC5

@misc{pith2026260802391,
  author       = {Pith},
  title        = {Pith review of: Cooperative Coevolution for Resource-Constrained Agentic LLM Post-Training},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/E43MWRC5}},
  note         = {Machine review of arXiv:2608.02391}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Tool-using large language model (LLM) agents produce long, multi-turn trajectories, making gradient-based post-training memory-intensive. Evolution strategies (ES) enable memory-efficient full-parameter post-training without backpropagation and can eventually match the performance of gradient-based reinforcement learning (RL). However, resource-constrained settings typically offer only a few GPUs, so the high GPU-hour requirements of ES translate into prohibitively long training times. To address this, we introduce Cooperative Parameter-subspace Evolution Strategy (CoPES), a cooperative coevolutionary method that decomposes the full parameter space into lower-dimensional subspaces and searches over them cooperatively to improve optimization efficiency. We post-train a Qwen3.5-4B tool-using agent for the math task and evaluate it on five benchmarks of varying difficulty. Under the GPU-hour budget of full-parameter GRPO's best validation checkpoint, CoPES recovers 92% of GRPO's validation-accuracy gain, versus 67% for standard ES, while its theoretical GPU memory requirement is less than one-eighth that of full-parameter GRPO. It consistently outperforms standard ES and LoRA-based GRPO on all evaluated pass@k metrics across the five benchmarks. Additional experiments further show the advantage of CoPES on the question-answering task. These results demonstrate an improved trade-off between memory requirements and training time for agentic LLM post-training under resource constraints. The code is open-sourced in https://github.com/MetaronWang/CoPES

Figures

Figures reproduced from arXiv: 2608.02391 by Haoze Lv, Hui Ouyang, Jiahao Wu, Ke Tang, Ning Lu, Shaofeng Zhang, Shengcai Liu, Zhiyuan Wang.

Figure 1
Figure 1. Figure 1: Overview of CoPES. At each training step, the model parameters are randomly partitioned into [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Theoretical GPU memory requirements versus con [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Pass@k of four post-training methods under the 16-step full-parameter GRPO budget, with Qwen3.5-4B and Qwen3.5- 9B without post-training as references. Method 1× 24GB 1× 48GB 4× 48GB 8× 48GB Full-param GRPO × × × √ LoRA-based GRPO × × × √ Standard ES / CoPES √ √ √ √ [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

38 extracted references · 1 canonical work pages

  1. [1]

    Zhenyu Yang and Ke Tang and Xin Yao , title =. Inf. Sci. , volume =. 2008 , doi =

    2008

  2. [2]

    Narasimhan and Yuan Cao , title =

    Shunyu Yao and Jeffrey Zhao and Dian Yu and Nan Du and Izhak Shafran and Karthik R. Narasimhan and Yuan Cao , title =. Proceedings of. 2023 , url =

    2023

  3. [3]

    Zhihong Shao and Peiyi Wang and Qihao Zhu and Runxin Xu and Junxiao Song and Xiao Bi and Haowei Zhang and Mingchuan Zhang and Y. K. Li and Y. Wu and Daya Guo , title =. CoRR , volume =. 2024 , doi =

    2024

  4. [4]

    Hu and Yelong Shen and Phillip Wallis and Zeyuan Allen-Zhu and Yuanzhi Li and Shean Wang and Lu Wang and Weizhu Chen , title =

    Edward J. Hu and Yelong Shen and Phillip Wallis and Zeyuan Allen-Zhu and Yuanzhi Li and Shean Wang and Lu Wang and Weizhu Chen , title =. Proceedings of. 2022 , url =

    2022

  5. [5]

    Proceedings of

    Vijay Anand Korthikanti and Jared Casper and Sangkug Lym and Lawrence McAfee and Michael Andersch and Mohammad Shoeybi and Bryan Catanzaro , title =. Proceedings of

  6. [6]

    CoRR , volume =

    Tim Salimans and Jonathan Ho and Xi Chen and Szymon Sidor and Ilya Sutskever , title =. CoRR , volume =. 2017 , url =

    2017

  7. [7]

    Hayes and Qiyao Liang and Yinggan Xu and Roberto Dailey and Elliot Meyerson and Babak Hodjat and Risto Miikkulainen , title =

    Xin Qiu and Yulu Gan and Conor F. Hayes and Qiyao Liang and Yinggan Xu and Roberto Dailey and Elliot Meyerson and Babak Hodjat and Risto Miikkulainen , title =. Proceedings of. 2026 , url =

    2026

  8. [8]

    CoRR , volume =

    WenTao Liu and Siyu Song and Hao Hao and Aimin Zhou , title =. CoRR , volume =. 2025 , doi =

    2025

  9. [9]

    CoRR , volume =

    Zhishen Sun and Sizhe Dang and Guang Dai and Haishan Ye , title =. CoRR , volume =. 2026 , doi =

    2026

  10. [10]

    Potter and Kenneth A

    Mitchell A. Potter and Kenneth A. A Cooperative Coevolutionary Approach to Function Optimization , booktitle =

  11. [11]

    Provable Space Discretization Based Evolutionary Search for Scalable Multi-Objective Security Games , journal =

    Hong Qian and Yu. Provable Space Discretization Based Evolutionary Search for Scalable Multi-Objective Security Games , journal =. 2025 , doi =

    2025

  12. [12]

    Proceedings of

    Caigao Jiang and Xiang Shu and Hong Qian and Xingyu Lu and Jun Zhou and Aimin Zhou and Yang Yu , title =. Proceedings of

  13. [13]

    The Landscape of Agentic Reinforcement Learning for LLMs:

    Guibin Zhang and Hejia Geng and Xiaohang Yu and Zhenfei Yin and Zaibin Zhang and Zelin Tan and Heng Zhou and Zhong. The Landscape of Agentic Reinforcement Learning for LLMs:. Trans. Mach. Learn. Res. , volume =. 2026 , url =

    2026

  14. [14]

    Proceedings of

    Bowen Jin and Hansi Zeng and Zhenrui Yue and Dong Wang and Hamed Zamani and Jiawei Han , title =. Proceedings of. 2025 , url =

    2025

  15. [15]

    CoRR , volume =

    Jiazhan Feng and Shijue Huang and Xingwei Qu and Ge Zhang and Yujia Qin and Baoquan Zhong and Chengquan Jiang and Jinxin Chi and Wanjun Zhong , title =. CoRR , volume =. 2025 , doi =

    2025

  16. [16]

    Evolution Strategies at the Hyperscale , booktitle =

    Bidipta Sarkar and Mattie Fellows and Juan Agustin Duque and Alistair Letcher and Antonio Le. Evolution Strategies at the Hyperscale , booktitle =. 2026 , url =

    2026

  17. [17]

    Potter and Kenneth A

    Mitchell A. Potter and Kenneth A. Cooperative Coevolution: An Architecture for Evolving Coadapted Subcomponents , journal =. 2000 , doi =

    2000

  18. [18]

    2014 , doi =

    Mohammad Nabi Omidvar and Xiaodong Li and Yi Mei and Xin Yao , title =. 2014 , doi =

    2014

  19. [19]

    Swarm Evol

    Peng Yang and Hu Zhang and Yanglong Yu and Mingjia Li and Ke Tang , title =. Swarm Evol. Comput. , volume =. 2022 , url =

    2022

  20. [20]

    2026 , month = feb, url =

    2026

  21. [21]

    Proceedings of

    Dan Hendrycks and Collin Burns and Saurav Kadavath and Akul Arora and Steven Basart and Eric Tang and Dawn Song and Jacob Steinhardt , title =. Proceedings of. 2021 , url =

    2021

  22. [22]

    CoRR , volume =

    Karl Cobbe and Vineet Kosaraju and Mohammad Bavarian and Mark Chen and Heewoo Jun and Lukasz Kaiser and Matthias Plappert and Jerry Tworek and Jacob Hilton and Reiichiro Nakano and Christopher Hesse and John Schulman , title =. CoRR , volume =. 2021 , url =

    2021

  23. [23]

    Proceedings of

    Hunter Lightman and Vineet Kosaraju and Yuri Burda and Harrison Edwards and Bowen Baker and Teddy Lee and Jan Leike and John Schulman and Ilya Sutskever and Karl Cobbe , title =. Proceedings of. 2024 , url =

    2024

  24. [24]

    Cohen and Ruslan Salakhutdinov and Christopher D

    Zhilin Yang and Peng Qi and Saizheng Zhang and Yoshua Bengio and William W. Cohen and Ruslan Salakhutdinov and Christopher D. Manning , title =. Proceedings of. 2018 , doi =

    2018

  25. [25]

    Proceedings of

    Xanh Ho and Anh-Khoa Duong Nguyen and Saku Sugawara and Akiko Aizawa , title =. Proceedings of. 2020 , doi =

    2020

  26. [26]

    Harsh Trivedi and Niranjan Balasubramanian and Tushar Khot and Ashish Sabharwal , title =. Trans. Assoc. Comput. Linguist. , volume =. 2022 , doi =

    2022

  27. [27]

    Evaluating Large Language Models Trained on Code , journal =

    Mark Chen and Jerry Tworek and Heewoo Jun and Qiming Yuan and Henrique Pond. Evaluating Large Language Models Trained on Code , journal =. 2021 , url =

    2021

  28. [28]

    Proceedings of the AAAI Symposium Series , volume =

    Uday Devulapalli and Aarat Satsangi and Apurva Narayan , title =. Proceedings of the AAAI Symposium Series , volume =. 2025 , doi =

    2025

  29. [29]

    Natural Evolution Strategies , journal =

    Daan Wierstra and Tom Schaul and Tobias Glasmachers and Yi Sun and Jan Peters and J. Natural Evolution Strategies , journal =. 2014 , url =

    2014

  30. [30]

    Dongfu Jiang and Yi Lu and Zhuofeng Li and Zhiheng Lyu and Ping Nie and Haozhe Wang and Alex Su and Hui Chen and Kai Zou and Chao Du and Tianyu Pang and Wenhu Chen , title =. Trans. Mach. Learn. Res. , year =

  31. [31]

    Proceedings of

    Chengpeng Hu and Jialin Liu and Xin Yao , title =. Proceedings of. 2024 , publisher =. doi:10.3233/FAIA240878 , url =

    work page doi:10.3233/faia240878 2024

  32. [32]

    Fu and Stefano Ermon and Atri Rudra and Christopher R

    Tri Dao and Daniel Y. Fu and Stefano Ermon and Atri Rudra and Christopher R. Proceedings of. 2022 , url =

    2022

  33. [33]

    Gonzalez and Hao Zhang and Ion Stoica , title =

    Woosuk Kwon and Zhuohan Li and Siyuan Zhuang and Ying Sheng and Lianmin Zheng and Cody Hao Yu and Joseph E. Gonzalez and Hao Zhang and Ion Stoica , title =. Proceedings of. 2023 , publisher =. doi:10.1145/3600006.3613165 , url =

    arXiv 2023

  34. [34]

    Proceedings of

    Guangming Sheng and Chi Zhang and Zilingfeng Ye and Xibin Wu and Wang Zhang and Ru Zhang and Yanghua Peng and Haibin Lin and Chuan Wu , title =. Proceedings of. 2025 , publisher =. doi:10.1145/3689031.3696075 , url =

    arXiv 2025

  35. [35]

    Smith and Mateusz Paprocki and Ond

    Aaron Meurer and Christopher P. Smith and Mateusz Paprocki and Ond. PeerJ Comput. Sci. , volume =. 2017 , doi =

    2017

  36. [36]

    CoRR , volume =

    Haoze Lv and Ning Lu and Ziang Zhou and Shengcai Liu , title =. CoRR , volume =. 2026 , url =

    2026

  37. [37]

    CoRR , volume =

    Jiahao Wu and Ning Lu and Shengcai Liu and Kun Wang and Yanting Yang and Li Qing and Ke Tang , title =. CoRR , volume =. 2026 , url =. doi:10.48550/ARXIV.2603.25184 , eprinttype =. 2603.25184 , timestamp =

    Pith/arXiv arXiv doi:10.48550/arxiv.2603.25184 2026

  38. [38]

    Safe Delta: Consistently Preserving Safety when Fine-Tuning

    Lu, Ning and Liu, Shengcai and Wu, Jiahao and Chen, Weiyu and Zhang, Zhirui and Ong, Yew-Soon and Wang, Qi and Tang, Ke , booktitle =. Safe Delta: Consistently Preserving Safety when Fine-Tuning. 2025 , url =

    2025