Pith. sign in

REVIEW 4 major objections 6 minor 61 references

Hyper-ES: Effective Evolution Strategies for LLM Reasoning via Descent Direction Merging

T0 review · 4 major / 6 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Short gradient runs define a subspace where evolution search beats longer single-run fine-tuning.

desk verdict Sensible method paper with a coherent pipeline and honest ablations, but the headline 1% gain over GRPO+LoRA is within noise; worth a serious referee, not acceptance as stated. read the letter →

arxiv 2608.05541 v1 pith:3V2WUZTW submitted 2026-08-06 cs.AI

Yu Gu , Zhi Zheng , Yunpeng Ba , Xialiang Tong , Mingxuan Yuan , Zhenkun Wang This is my paper · ORCID
classification cs.AI
keywords evolutionstrategiesCMA-ESLLMreasoningmodelmergingDARE-TIESGRPOLoRAmathematical
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that evolution strategies can be made effective for fine-tuning billion-parameter LLMs if the search is confined to the low-dimensional span of a few short gradient-based runs. The proposed method, Hyper-ES, runs several brief GRPO updates on difficulty-grouped data subsets to produce LoRA descent directions, then uses CMA-ES to optimize layer-wise DARE-TIES merging coefficients over these directions. If correct, the span of short gradient runs contains better models than the endpoint of a longer single GRPO run, so reasoning fine-tuning can be both slightly more accurate and roughly 10% cheaper in gradient-requiring samples. A sympathetic reader would care because this turns an unstable full-space black-box search into a small coefficient search while retaining the memory-light property of evolution strategies.

What carries the argument

The central object is the descent direction pool P = {Δθ_i^(ℓ)} built from N few-shot (7-step) GRPO LoRA runs on difficulty-grouped subsets, with each delta computed as (α/r) B_i A_i without materializing a full model. CMA-ES then searches a vector z of size L×2N (here 960 for the 0.5B model) containing, per layer group, DARE drop-rate logits and non-negative mixing-weight logits; decoding applies DARE sparsification, weighted TIES sign election and disjoint merge, and a validation-selected global scale γ to assemble θ*(z). The machinery replaces full-parameter direction discovery with low-dimensional direction selection: the span must be task-aligned (Proposition 2 formalizes the benefit when the projection cosine c>0), and the fitness steering CMA-ES is 600-sample validation accuracy.

What would settle it

Compare, across several seeds, the validation accuracy of the Hyper-ES merged model with that of a single GRPO+LoRA run continued until it has consumed the same total number of gradient-requiring samples on the same data; the central claim is settled by whether the merged model's accuracy exceeds the single run's.

Watch

Extended reading notes

Core claim

On three backbones across six math reasoning benchmarks, Hyper-ES consistently outperforms single-stage GRPO+LoRA by about 1% average accuracy (57.13 vs 56.23 for Qwen2.5-0.5B; 74.26 vs 73.51 for Qwen2.5-1.5B; 70.97 vs 70.39 for DeepSeek-R1-Distill-Qwen-1.5B) while using 17,920 versus 20,000 gradient-requiring samples. The discovery is that the span S = span{Δθ_1,...,Δθ_N} of short GRPO LoRA deltas, searched via layer-wise merging coefficients, carries enough task-aligned signal that evolution strategy over combinations of these directions beats the model produced by a longer single gradient run; random directions in the same search space collapse to near-base performance (47.58), showing the deltas themselves, not low dimensionality alone, are what make the search work.

Load-bearing premise

The short GRPO runs produce LoRA deltas whose span has a positive projection onto the true loss-reducing direction (c > 0), and the 600-sample validation accuracy is a reliable enough fitness signal that CMA-ES's coefficient search is not chasing noise.

Editorial extensions

If this is right

  • GRPO-style gradient updates can serve as direction providers rather than final trainers, making short runs reusable building blocks.
  • ES retains its memory-light, parallelizable character while avoiding the high-dimensional orthogonality failure documented in Lemmas 1–2.
  • Fewer gradient-requiring samples suffice: 7-step directions use 17,920 samples and beat the 20,000-sample full GRPO run; 4-step directions still reach 55.71%.
  • Layer-wise, direction-specific merging coefficients beat uniform merging (Average Merge 55.64 vs Hyper-ES 57.13 on Qwen2.5-0.5B) and grid-only merging (55.92).
  • The same direction-pool search also improves out-of-domain code generation on Qwen2.5-1.5B (38.44 to 46.45), suggesting the mechanism is not specific to math.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: If the finding holds generally, the last part of RL fine-tuning may be better spent selecting a combination of short gradient runs than extending one run; a testable consequence is that a larger pool of even shorter or fewer-step runs could continue to improve accuracy.
  • Editorial inference: Any cheap gradient signal, such as SFT or other policy-gradient variants, could plausibly seed the direction pool; the paper tests only GRPO, leaving this as an open extension.
  • Editorial inference: A sharper comparison would pit Hyper-ES against GRPO+LoRA at equal wall-clock time rather than equal gradient samples, since the paper's parallelized 3.23 h already undercuts the 4.25 h baseline but is not the headline comparison.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. This paper proposes Hyper-ES, a two-stage fine-tuning method for LLM reasoning. In the first stage, the method partitions the training data into N subsets and runs a short GRPO+LoRA fine-tuning run (7 steps) on each subset, caching the resulting LoRA deltas as descent directions. In the second stage, the base model and the direction pool are frozen, and CMA-ES optimizes a vector z of layer-wise DARE-TIES merging coefficients (drop-rate logits and mixing-weight logits), with fitness defined as accuracy on a 600-question validation set. The paper reports results on Qwen2.5-0.5B-Instruct, Qwen2.5-1.5B-Instruct, and DeepSeek-R1-Distill-Qwen-1.5B across six math benchmarks, claiming an average improvement of about 1% over GRPO+LoRA while using 17,920 instead of 20,000 gradient-requiring samples. The paper also provides Lemmas 1 and 2 on the orthogonality and random-walk failure of direct ES in high dimensions, and Proposition 2 formalizes the task-alignment condition under which searching in the direction-pool span is beneficial.

Significance. The idea of using a few short gradient runs to define a low-dimensional subspace and then applying CMA-ES to merge those directions is interesting and, if confirmed, would have practical value for resource-constrained fine-tuning: it would show that the span of short task vectors can contain models that outperform the endpoint of a longer gradient run. The paper ships code and includes ablations that support the proposed mechanism: removing CMA-ES (Table 3, -1.21 points), removing difficulty grouping (Table 3, -3.51 points), and replacing GRPO directions with random directions (Table 4, 47.58 vs 57.13) all degrade performance. The geometric lemmas are standard but clearly stated. However, the headline empirical claim is not yet statistically established; the reported margins are comparable to or smaller than the noise level of the evaluation, and the multi-seed experiment in Appendix E.4.1 does not show a significant difference.

major comments (4)
  1. [§4.2–4.3, Tables 1–2; Appendix E.4.1, Table 11] The central claim that Hyper-ES 'consistently outperforms GRPO-LoRA by 1%' is not supported at conventional significance levels. Tables 1 and 2 report single-run test accuracies without variance; the average margins are 0.90, 0.75, and 0.58 points across the three backbones. The only repeated-seed experiment (Table 11, n=3) gives Hyper-ES 56.62±0.44 versus GRPO+LoRA 56.04±0.29; the difference of 0.58 has a pooled standard error of roughly 0.30, so it is not significant (t≈1.9, p>0.05), and the in-domain GSM8K result is numerically worse (50.85 vs 50.92). Please report per-benchmark confidence intervals over at least five independent runs and a formal comparison (e.g., paired test or mixed-effects model) before claiming consistent improvement.
  2. [§3.2, Eq. (7); Appendix D.2] The fitness function is accuracy on a 600-question validation set, whose binomial standard error is about 2.0 percentage points at p≈0.5. This is larger than each of the reported average gains (0.90, 0.75, 0.58). Furthermore, Appendix D.2 shows that the global scale and drop rate are selected by grid search on this same validation set, and CMA-ES then optimizes against the same set. With no separate validation split or nested cross-validation, the reported test improvements may partly reflect overfitting to validation noise rather than a genuine advantage. Please add an independent validation set for model selection, or show that the chosen configuration generalizes across multiple seeds and validation splits.
  3. [Appendix E.2, Table 9] The wall-clock efficiency claim is based on a partly extrapolated estimate: the CMA-ES stage is listed as '50 gens, extrapolated,' and the cumulative (non-parallelized) total of 7.39 h is 74% longer than the GRPO+LoRA baseline's 4.25 h. Only the parallelized total (3.23 h) supports the advertised 24% reduction, and that total inherits the extrapolated CMA-ES figure. Please report measured end-to-end times for all stages and clarify how the extrapolation was computed; otherwise the efficiency comparison is not verifiable.
  4. [Appendix C.1, Proposition 2] Proposition 2 assumes the projection of the true descent direction g* onto the direction-pool span has cosine c>0, but the paper does not directly measure or otherwise verify this alignment for the GRPO-derived directions; it is asserted that 'GRPO-derived task vectors are intended to provide this alignment.' The random-direction ablation (Table 4) shows that when alignment fails the method collapses, so this condition is load-bearing. Please provide a direct diagnostic of task alignment, for example the cosine between the aggregate GRPO update and the direction-pool span, or an ablation that varies the number of GRPO steps and reports the resulting alignment and end performance.
minor comments (6)
  1. [Abstract] The abstract states an improvement of '1%' while the average margins in Tables 1 and 2 are 0.90, 0.75, and 0.58 points; please report exact margins or use a more precise rounding convention.
  2. [Figure 2 caption and Appendix C.2] The caption says raw data are provided in Appendix C.2, but Appendix C.2 reports only aggregate cosine-similarity and norm tables (Tables 5 and 6), not the per-step raw data referenced; please either add the raw data or adjust the caption.
  3. [Lemma 2, §2.2 and Appendix C.1] Lemma 2's statement omits the cross-term uncorrelatedness assumption E⟨r_t, r_s⟩≈0 that appears in the proof; the lemma should state this assumption explicitly in the main text.
  4. [Tables 1 and 2] The benchmark name 'SV AMP' should be spelled 'SVAMP' to match the standard reference (Patel et al., 2021).
  5. [Appendix E.2, Table 9] The 'Cumulative' row in Table 9 sums per-direction GPU time and may be misleading without specifying that it is a single-GPU serial equivalent; please label it explicitly.
  6. [Appendix F.2] GSM8K-Aug and DeepScaler datasets are referenced via URLs but without formal bibliographic entries; please add proper references.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the derivation chain is self-contained and the central claim is an empirical comparison, not a construction-level reduction.

full rationale

The paper's derivation chain is a constructive pipeline: short GRPO runs define LoRA task vectors, CMA-ES searches layer-wise DARE-TIES coefficients, and the decoded merged model is evaluated on held-out test sets. No load-bearing step reduces, by the paper's own equations, to its inputs. Lemmas 1 and 2 are standard concentration and random-walk facts with explicit assumptions (log G = o(d), mean-zero orthogonal components) and are proved locally rather than imported from the authors' prior work. Proposition 2 is explicitly conditional: it formalizes the benefit of alignment c > 0 but does not assert that GRPO deltas satisfy it; the paper tests this alignment empirically through the random-direction ablation (Table 4), so it is not a self-definitional assumption. The global scale and drop rate are selected on a 600-question validation set and then applied to test benchmarks; this is ordinary hyperparameter selection, not a fitted parameter renamed as a prediction. Self-citations (Soft-GRPO; Zheng and Lee 2025 and 2026) appear only as references to GRPO-family methods in related work and do not carry the main argument. The only substantive concern is statistical: the reported 1% gains are close to measurement noise and the multi-seed appendix shows a gap that is not clearly significant. That is a correctness or robustness issue, not circularity, and therefore does not raise the circularity score.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The ledger counts six hand-chosen or validation-fitted hyperparameters on which the reported comparison depends. The two most load-bearing are the global merge scale and the DARE drop rate, both selected by grid search on the same 600-question validation set that also serves as the CMA-ES fitness, creating a mild selection-on-validation risk. The method also rests on five assumptions, of which the task-alignment of short-GRPO direction subspaces (Proposition 2) is the most important, since the random-direction ablation shows the whole gain disappears when it fails. Lemma 2 additionally assumes uncorrelated, mean-zero orthogonal ES updates, which is stated in the proof sketch rather than established. No new physical or conceptual entities are postulated beyond the algorithmic constructs of a direction pool and coefficient search space.

free parameters (6)
  • Global merge scale (gamma = scale x N) = 2.7 (Qwen2.5-0.5B), 2.6 (Qwen2.5-1.5B), 2.9 (DeepSeek-R1-Distill-1.5B)
    Grid-searched over scale in {1.0, ..., 3.0} on the 600-sample validation set (Appendix D.2); directly scales the merged update and affects all reported accuracies.
  • DARE drop rate p = 0.9 (0.5B), 0.7 (1.5B), 0.5 (DeepSeek)
    Grid-searched over {0.3, 0.5, 0.7, 0.9} on validation; controls direction sparsification and is fit per model.
  • Number of directions N = 10 (Qwen2.5), 7 (DeepSeek)
    Hand-chosen; sets the search dimension L x 2N (960 for 0.5B) and the total gradient budget (17,920 vs 12,544 samples).
  • GRPO steps per direction = 7 (4 in ablation)
    Hand-chosen trade-off; the 4-step ablation drops average accuracy from 57.13 to 55.71.
  • LoRA rank and scale = r=32, alpha=64 (alpha/r = 2.0)
    Fixed across all models; determines the norm of each direction vector entering the merge.
  • CMA-ES initial step size and population size = sigma_z = 0.3; G = 4 + floor(3 ln(960)) ≈ 24
    Hand-chosen; population size grows logarithmically with search dimension per CMA-ES practice.
assumptions (5)
  • standard math Concentration of measure on the sphere: for a uniformly sampled unit vector u in R^d, Pr(|<u,e1>| >= tau) <= 2 exp(-d tau^2 / 2)
    Used in the proof of Lemma 1 (Appendix C.1) to justify near-orthogonality of random perturbations in high dimensions. Standard result; not in dispute.
  • ad hoc to paper Orthogonal components of ES updates are mean-zero and uncorrelated across iterations, E<r_t, r_s> ≈ 0 for t != s
    Explicitly assumed in the proof of Lemma 2 (Appendix C.1). The random-walk drift bound in Eq. (14) depends on this assumption; it is a modeling choice rather than a proven property of ES.
  • domain assumption Short GRPO runs on difficulty-grouped subsets produce LoRA deltas whose span S has positive cosine alignment c > 0 with the true descent direction g*
    Proposition 2 (Appendix C.1) states the benefit of alignment but does not prove the alignment exists; it is the load-bearing premise of the method (Section 3.1). The pure-random ablation supports it empirically for this setting.
  • domain assumption Validation accuracy on a fixed 600-question set is a reliable fitness signal for CMA-ES and does not overfit the merging coefficients to the validation data
    Eq. (7) defines the objective as validation accuracy; with 600 items the binomial standard error is roughly 2 points, comparable to the reported gains, so this assumption is fragile and enters Sections 3.2 and 4.1.
  • domain assumption GRPO+LoRA is a fairly tuned representative of gradient-based RLVR fine-tuning
    The central comparison in Tables 1 and 2 is against this baseline; hyperparameters are given but no tuning curve or significance test, so the 0.6 to 0.9 point advantage could be baseline-dependent. Enters Section 4.1.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Hyper-ES: Effective Evolution Strategies for LLM Reasoning via Descent Direction Merging." pith.science (2026). https://pith.science/paper/3V2WUZTW

@misc{pith2026260805541,
  author       = {Pith},
  title        = {Pith review of: Hyper-ES: Effective Evolution Strategies for LLM Reasoning via Descent Direction Merging},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3V2WUZTW}},
  note         = {Machine review of arXiv:2608.05541}
}
read the original abstract

Evolution Strategy (ES) is a promising alternative to gradient-based fine-tuning for resource-constrained Large Language Model (LLM) reasoning. However, directly applying ES to billion-parameter LLMs is highly ineffective. In such high-dimensional parameter spaces, most random perturbations are nearly orthogonal to useful update directions, leading to unstable optimization. We propose Hyper-ES, a subspace-based ES framework that avoids the weakness of ES in full-parameter search while exploiting its strength in low-dimensional optimization. Instead of asking ES to discover useful directions from random perturbations in the LLM parameter space, Hyper-ES first performs a small number of inexpensive gradient-based fine-tuning runs to obtain descent directions. Although each direction may provide only a limited improvement on its own, their span forms a compact adaptation subspace that captures useful reasoning updates. Hyper-ES then applies CMA-ES to optimize layer-wise DARE-TIES merging coefficients within this subspace, allowing ES to search over combinations of meaningful descent directions rather than over arbitrary full-model perturbations. We evaluate Hyper-ES on three Qwen2.5-Instruct and DeepSeek-R1-Distill backbones across six mathematical reasoning datasets. Results show that Hyper-ES consistently outperforms GRPO-LoRA by 1% while requiring 10% fewer space-consuming gradient updates. Code at https://github.com/kuangrepi/Hyper-ES.

Figures

Figures reproduced from arXiv: 2608.05541 by the authors.

Figure 1
Figure 1. (a) GRPO updates model parameters with policy gradients, which is usually resource-consuming. (b) ES offers a gradient-free method for LLM reason￾ing, suitable for resource-constrained scenarios, but will lead to significant inefficiency & out-of-control random walk. (c) Hyper-ES replaces full-parameter exploration with low-dimensional search over coefficients of fast￾obtained descent directions, leading to Higher L… view at source ↗
Figure 2
Figure 2. Evidence for Lemmas 1 and 2. We show the detailed raw data for this figure in Appendix C.2. Lemma 2 (Irrelevant ES updates accumulate pa￾rameter drift). Let rt = gbES(θt) − Projg ∗ gbES(θt) denote the component of the ES update estimate orthogonal to g ∗ . Suppose the useful component is negligible and the orthogonal components have a nonzero second moment, E∥rt∥ 2 2 = ρ 2 > 0. Then, after T ES steps, the accumulate… view at source ↗
Figure 3
Figure 3. Implementation details of HYPER-ES. (a) A small number of few-shot GRPO updates extract task-relevant descent directions from different data subsets. (b) A search variable z is decoded into layer-wise DARE–TIES merging coefficients that combine these directions into a final model. layer group, z contains two types of variables: drop-rate logits controlling DARE (Yu et al., 2024) sparsification, and mixing-weight log… view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Sample-efficiency curve on Qwen2.5-0.5B [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

61 extracted references · 14 canonical work pages

  1. [1]

    arXiv preprint arXiv:2212.04089 , year=

    Editing models with task arithmetic , author=. arXiv preprint arXiv:2212.04089 , year=

    arXiv

  2. [2]

    arXiv preprint arXiv:2601.21598 , year=

    Beyond Imitation: Reinforcement Learning for Active Latent Planning , author=. arXiv preprint arXiv:2601.21598 , year=

    work page arXiv

  3. [3]

    arXiv preprint arXiv:2511.06411 , year=

    Soft-grpo: Surpassing discrete-token llm reinforcement learning via gumbel-reparameterized soft-thinking policy optimization , author=. arXiv preprint arXiv:2511.06411 , year=

    arXiv

  4. [4]

    Advances in neural information processing systems , volume=

    Ties-merging: Resolving interference when merging models , author=. Advances in neural information processing systems , volume=

  5. [5]

    arXiv preprint arXiv:2503.20783 , year=

    Understanding r1-zero-like training: A critical perspective , author=. arXiv preprint arXiv:2503.20783 , year=

    arXiv

  6. [6]

    arXiv preprint arXiv:2510.10603 , year=

    EA4LLM: A Gradient-Free Approach to Large Language Model Optimization via Evolutionary Algorithms , author=. arXiv preprint arXiv:2510.10603 , year=

    arXiv

  7. [7]

    arXiv preprint arXiv:2603.12228 , year=

    Neural thickets: Diverse task experts are dense around pretrained weights , author=. arXiv preprint arXiv:2603.12228 , year=

    arXiv

  8. [8]

    Reasoning Resides in Layers: Restoring Temporal Reasoning in Video-Language Models with Layer-Selective Merging

    Reasoning Resides in Layers: Restoring Temporal Reasoning in Video-Language Models with Layer-Selective Merging , author=. arXiv preprint arXiv:2604.11399 , year=

    work page Pith review arXiv

Show all 61 references
  1. [9]

    arXiv preprint arXiv:2502.17419 , year=

    From system 1 to system 2: A survey of reasoning large language models , author=. arXiv preprint arXiv:2502.17419 , year=

    arXiv

  2. [10]

    Forty-first International Conference on Machine Learning , year=

    Language models are super mario: Absorbing abilities from homologous models as a free lunch , author=. Forty-first International Conference on Machine Learning , year=

  3. [11]

    Nature Machine Intelligence , volume=

    Evolutionary optimization of model merging recipes , author=. Nature Machine Intelligence , volume=. 2025 , publisher=

    2025

  4. [12]

    arXiv preprint arXiv:2509.24372 , year=

    Evolution strategies at scale: Llm fine-tuning beyond reinforcement learning , author=. arXiv preprint arXiv:2509.24372 , year=

    arXiv

  5. [13]

    arXiv preprint arXiv:2402.03300 , year=

    Deepseekmath: Pushing the limits of mathematical reasoning in open language models , author=. arXiv preprint arXiv:2402.03300 , year=

    arXiv

  6. [14]

    , author=

    Lora: Low-rank adaptation of large language models. , author=. Iclr , volume=

  7. [15]

    arXiv preprint arXiv:2110.14168 , year=

    Training verifiers to solve math word problems , author=. arXiv preprint arXiv:2110.14168 , year=

    arXiv

  8. [16]

    2025 , eprint=

    Qwen2.5 Technical Report , author=. 2025 , eprint=

    2025

  9. [17]

    Kydlíček, Hynek , license =

  10. [18]

    arXiv preprint arXiv:2604.01499 , year=

    Matching Accuracy, Different Geometry: Evolution Strategies vs GRPO in LLM Post-Training , author=. arXiv preprint arXiv:2604.01499 , year=

  11. [19]

    arXiv preprint arXiv:2501.12948 , year=

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning , author=. arXiv preprint arXiv:2501.12948 , year=

    arXiv

  12. [20]

    Notion Blog , volume=

    Deepscaler: Surpassing o1-preview with a 1.5 b model by scaling rl , author=. Notion Blog , volume=

  13. [21]

    arXiv preprint arXiv:2508.08221 , year=

    Part i: Tricks or traps? a deep dive into rl for llm reasoning , author=. arXiv preprint arXiv:2508.08221 , year=

  14. [22]

    URL https://arxiv

    Dapo: An open-source llm reinforcement learning system at scale, 2025 , author=. URL https://arxiv. org/abs/2503.14476 , volume=

    2025 arXiv

  15. [23]

    arXiv preprint arXiv:2602.02710 , year=

    Maximum Likelihood Reinforcement Learning , author=. arXiv preprint arXiv:2602.02710 , year=

  16. [24]

    arXiv preprint arXiv:2512.23165 , year=

    Evaluating Parameter Efficient Methods for RLVR , author=. arXiv preprint arXiv:2512.23165 , year=

  17. [25]

    arXiv preprint arXiv:2505.05464 , year=

    Bring reason to vision: Understanding perception and reasoning through model merging , author=. arXiv preprint arXiv:2505.05464 , year=

    arXiv

  18. [26]

    arXiv preprint arXiv:2510.08987 , year=

    Towards Efficient Multimodal Unified Reasoning Model via Model Merging , author=. arXiv preprint arXiv:2510.08987 , year=

  19. [27]

    arXiv preprint arXiv:2601.20861 , year=

    Evolutionary Strategies lead to Catastrophic Forgetting in LLMs , author=. arXiv preprint arXiv:2601.20861 , year=

  20. [28]

    arXiv preprint arXiv:1703.03864 , year=

    Evolution strategies as a scalable alternative to reinforcement learning , author=. arXiv preprint arXiv:1703.03864 , year=

    arXiv

  21. [29]

    arXiv preprint arXiv:2602.01003 , year=

    ESSAM: A Novel Competitive Evolution Strategies Approach to Reinforcement Learning for Memory Efficient LLMs Fine-Tuning , author=. arXiv preprint arXiv:2602.01003 , year=

    arXiv

  22. [30]

    arXiv preprint arXiv:2511.16652 , year=

    Evolution strategies at the hyperscale , author=. arXiv preprint arXiv:2511.16652 , year=

  23. [31]

    arXiv preprint arXiv:2408.01800 , year=

    Minicpm-v: A gpt-4v level mllm on your phone , author=. arXiv preprint arXiv:2408.01800 , year=

    arXiv

  24. [32]

    Advances in neural information processing systems , volume=

    Chain-of-thought prompting elicits reasoning in large language models , author=. Advances in neural information processing systems , volume=

  25. [33]

    arXiv preprint arXiv:2503.23077 , year=

    Efficient inference for large reasoning models: A survey , author=. arXiv preprint arXiv:2503.23077 , year=

    arXiv

  26. [34]

    arXiv preprint arXiv:2505.12348 , year=

    Reasoning-CV: Fine-tuning Powerful Reasoning LLMs for Knowledge-Assisted Claim Verification , author=. arXiv preprint arXiv:2505.12348 , year=

    arXiv

  27. [35]

    arXiv preprint arXiv:2507.01079 , year=

    MobileRAG: A Fast, Memory-Efficient, and Energy-Efficient Method for On-Device RAG , author=. arXiv preprint arXiv:2507.01079 , year=

    arXiv

  28. [36]

    Proceedings of the 26th annual international conference on machine learning , pages=

    Curriculum learning , author=. Proceedings of the 26th annual international conference on machine learning , pages=

  29. [37]

    Proceedings of the 62nd annual meeting of the association for computational linguistics (volume 3: system demonstrations) , pages=

    Llamafactory: Unified efficient fine-tuning of 100+ language models , author=. Proceedings of the 62nd annual meeting of the association for computational linguistics (volume 3: system demonstrations) , pages=

  30. [38]

    International Conference on Learning Representations , volume=

    Let's verify step by step , author=. International Conference on Learning Representations , volume=

  31. [39]

    Proceedings of the 2021 conference of the North American chapter of the association for computational linguistics: human language technologies , pages=

    Are NLP models really able to solve simple math word problems? , author=. Proceedings of the 2021 conference of the North American chapter of the association for computational linguistics: human language technologies , pages=

    2021

  32. [40]

    Proceedings of the 2015 conference on empirical methods in natural language processing , pages=

    Solving general arithmetic word problems , author=. Proceedings of the 2015 conference on empirical methods in natural language processing , pages=

    2015

  33. [41]

    International conference on machine learning , pages=

    Pal: Program-aided language models , author=. International conference on machine learning , pages=. 2023 , organization=

    2023

  34. [42]

    Evolutionary computation , volume=

    Completely derandomized self-adaptation in evolution strategies , author=. Evolutionary computation , volume=. 2001 , publisher=

    2001

  35. [43]

    arXiv preprint arXiv:2409.19256 , year=

    Hybridflow: A flexible and e! cient rlhf framework , author=. arXiv preprint arXiv:2409.19256 , year=

    arXiv

  36. [44]

    arXiv preprint arXiv:2512.05760 , year=

    Evolutionary System 2 Reasoning: An Empirical Proof , author=. arXiv preprint arXiv:2512.05760 , year=

  37. [45]

    arXiv preprint arXiv:2507.04453 , year=

    ESSA: Evolutionary Strategies for Scalable Alignment , author=. arXiv preprint arXiv:2507.04453 , year=

  38. [46]

    ACM Computing Surveys , volume=

    Model merging in llms, mllms, and beyond: Methods, theories, applications, and opportunities , author=. ACM Computing Surveys , volume=. 2026 , publisher=

    2026

  39. [47]

    arXiv preprint arXiv:2307.13269 , year=

    Lorahub: Efficient cross-task generalization via dynamic lora composition , author=. arXiv preprint arXiv:2307.13269 , year=

    arXiv

  40. [48]

    arXiv preprint arXiv:2405.14838 , year=

    From explicit cot to implicit cot: Learning to internalize cot step by step , author=. arXiv preprint arXiv:2405.14838 , year=

    arXiv

  41. [49]

    arXiv preprint arXiv:2503.01874 , year=

    Cabs: Conflict-aware and balanced sparsification for enhancing model merging , author=. arXiv preprint arXiv:2503.01874 , year=

    arXiv

  42. [50]

    arXiv preprint arXiv:2503.08099 , year=

    Whoever started the interference should end it: Guiding data-free model merging via task vectors , author=. arXiv preprint arXiv:2503.08099 , year=

    arXiv

  43. [51]

    arXiv preprint arXiv:2501.08603 , year=

    Monte carlo tree search for comprehensive exploration in llm-based automatic heuristic design , author=. arXiv preprint arXiv:2501.08603 , year=

    arXiv

  44. [52]

    Advances in Neural Information Processing Systems , volume=

    UDC: A unified neural divide-and-conquer framework for large-scale combinatorial optimization problems , author=. Advances in Neural Information Processing Systems , volume=

  45. [53]

    arXiv preprint arXiv:2412.00346 , year=

    CaDA: Cross-problem routing solver with constraint-aware dual-attention , author=. arXiv preprint arXiv:2412.00346 , year=

    arXiv

  46. [54]

    arXiv preprint arXiv:2405.17272 , year=

    DPN: Decoupling partition and navigation for neural solvers of min-max vehicle routing problems , author=. arXiv preprint arXiv:2405.17272 , year=

    arXiv

  47. [55]

    Proceedings of the AAAI Conference on Artificial Intelligence , volume=

    Learning encodings for constructive neural combinatorial optimization needs to regret , author=. Proceedings of the AAAI Conference on Artificial Intelligence , volume=

  48. [56]

    IEEE Transactions on Intelligent Transportation Systems , volume=

    Pareto improver: Learning improvement heuristics for multi-objective route planning , author=. IEEE Transactions on Intelligent Transportation Systems , volume=. 2023 , publisher=

    2023

  49. [57]

    Advances in Neural Information Processing Systems , volume=

    Rethinking neural combinatorial optimization for vehicle routing problems with different constraint tightness degrees , author=. Advances in Neural Information Processing Systems , volume=

  50. [58]

    2026 , eprint=

    One Token per Multimodal Evidence: Latent Memory for Resource-Constrained QA , author=. 2026 , eprint=

    2026

  51. [59]

    2026 , eprint=

    APEX: Autonomous Policy Exploration for Self-Evolving LLM Agents , author=. 2026 , eprint=

    2026

  52. [60]

    arXiv preprint arXiv:2108.07732 , year=

    Program synthesis with large language models , author=. arXiv preprint arXiv:2108.07732 , year=

    arXiv

  53. [61]

    arXiv preprint arXiv:2107.03374 , year=

    Evaluating large language models trained on code , author=. arXiv preprint arXiv:2107.03374 , year=

    arXiv

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.