Artificial Intelligence · arXiv:2602.14697 v5 · submitted 16 Feb 2026
Evolutionary System Prompt Learning for Reinforcement Learning in LLMs
The paper says
“Across all settings, E-SPL outperforms both RL-only and evolution-only baselines, demonstrating that weight updates and context updates are deeply synergistic and can together yield gains in generalization that neither achieves alone.”
from the abstract · picked out as the main claim by a classifier
Reviews
Reviews by people
No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.
The paper
Abstract
Building agentic systems that can autonomously self-improve from experience is a longstanding goal of AI. Large language models (LLMs) today primarily self-improve via two mechanisms: self-reflection for context updates, and reinforcement learning (RL) for weight updates. In this work, we propose Evolutionary System Prompt Learning (E-SPL), a method for jointly improving a model's experiential context and its weights. In each RL iteration, E-SPL samples trajectories under multiple system prompts in parallel, then uses them to jointly update the weights with RL and evolve the system prompts with self-reflection. E-SPL encourages a natural division between procedural knowledge in weights and interpretable declarative knowledge in system prompts. Most notably, in an easy-to-hard generalization setting (AIME → BeyondAIME), E-SPL improves RL success rate from 38.8% → 45.1%. E-SPL also improves RL on AIME 2025 (56.3% → 60.6%), HMMT 2025 (50.0% → 52.7%), and agentic search (44.2% → 48.6% on gpt-oss-120b). Across all settings, E-SPL outperforms both RL-only and evolution-only baselines, demonstrating that weight updates and context updates are deeply synergistic and can together yield gains in generalization that neither achieves alone. Code is available at https://github.com/LunjunZhang/E-SPL.
The paper's own sentences, arranged. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.
abstract compiled · 2026-11-05 · no language-model review
How the main claim is stated: asserted 1.00
What the abstract itself states, as probabilities: claims first 0.06 · states limitation 0.06 · human participants 0.03 · controlled comparison 0.96 · randomized assignment 0.02 · states quantitative result 0.99 · compares with prior results 0.97 · mentions code or data availability 0.99
The abstract, sentence by sentence
This table labels the abstract only (arXiv listing abstract v4). The number is the probability the classifier gave the label.
| Sentence | Role | The authors' own work | States the main conclusion | Strength |
|---|---|---|---|---|
| Building agentic systems that can autonomously self-improve from experience is a longstanding goal of AI. | background 1.00 | 0.03 | 0.02 | asserted 1.00 |
| Large language models (LLMs) today primarily self-improve via two mechanisms: self-reflection for context updates, and reinforcement learning (RL) for weight updates. | background 1.00 | 0.06 | 0.02 | asserted 1.00 |
| In this work, we propose Evolutionary System Prompt Learning (E-SPL), a method for jointly improving a model's experiential context and its weights. | contribution 0.97 | 0.74 | 0.08 | asserted 0.97 |
| In each RL iteration, E-SPL samples trajectories under multiple system prompts in parallel, then uses them to jointly update the weights with RL and evolve the system prompts with self-reflection. | method 1.00 | 0.87 | 0.05 | asserted 0.99 |
| E-SPL encourages a natural division between procedural knowledge in weights and interpretable declarative knowledge in system prompts. | interpretation 0.71 | 0.51 | 0.06 | asserted 0.84 |
| Most notably, in an easy-to-hard generalization setting (AIME $\rightarrow$ BeyondAIME), E-SPL improves RL success rate from 38.8% $\rightarrow$ 45.1%. | result 1.00 | 0.95 | 0.13 | asserted 1.00 |
| E-SPL also improves RL on AIME 2025 (56.3% $\rightarrow$ 60.6%), HMMT 2025 (50.0% $\rightarrow$ 52.7%), and agentic search (44.2% $\rightarrow$ 48.6% on gpt-oss-120b). | result 1.00 | 0.95 | 0.06 | asserted 1.00 |
| Across all settings, E-SPL outperforms both RL-only and evolution-only baselines, demonstrating that weight updates and context updates are deeply synergistic and can together yield gains in generalization that neither achieves alone. | interpretation 0.70 | 0.94 | 0.90 | asserted 1.00 |
| Code is available at https://github.com/LunjunZhang/E-SPL. | contribution 0.70 | 0.44 | 0.02 | asserted 0.75 |
What kind of paper the classifier reports this is: method or tool 1.00 · evidence from observation or experiment 0.92
What was examined, and what was not
Sources read
- arXiv listing abstract v4
Not examined
- full text
- figures and tables
- references and citing sentences
- relations to other papers
Levels run
- P abstract
Not run
- S sections
- T body sentences
- C citations
- R cross-paper relations
How the labels were made
Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-11-05: 1 request, 7,134 input tokens.
Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON
How the literature uses it
2 arXiv papers cite this in 2 sentences
Papers citing it
Compares with it 2 papers
-
Recent work E-SPL [ref] explores prompt optimization and RL at a smaller scale with focus on performance rather than adaptation. mmGRPO [ref] runs prompt optimization once and then RL on a DSPy program; we interleave the two across cycles.
Learning, Fast and Slow: Towards LLMs That Adapt Continually · Related Work · 0.95
-
E-SPL jointly optimizes a global free-text prompt and a policy from shared rollouts [ref] . instead combines validated evolution of structured workflow context with reward-based LoRA adaptation.
CoAdapt-GUI: Joint Workflow Context and Policy Adaptation for Unseen GUI Applications · Test-Time Context and Policy Adaptation · 0.93
Challenges and discussion
Know something about this paper? Leave a comment.
Come back by email
A short list of the week's newly reviewed papers. Nothing is sent until you confirm from your inbox, and every email has a link that stops it. No account.
No comments yet.
Sign in to comment