Promptbreeder evolves both task prompts and the mutation prompts that improve them using LLMs, outperforming Chain-of-Thought and Plan-and-Solve on arithmetic and commonsense reasoning benchmarks.
hub
Population Based Training of Neural Networks
37 Pith papers cite this work, alongside 250 external citations. Polarity classification is still indexing.
37
Pith papers citing it
250
external citations · Pith
abstract
Neural networks dominate the modern machine learning landscape, but their training and success still suffer from sensitivity to empirical choices of hyperparameters such as model architecture, loss function, and optimisation algorithm. In this work we present \emph{Population Based Training (PBT)}, a simple asynchronous optimisation algorithm which effectively utilises a fixed computational budget to jointly optimise a population of models and their hyperparameters to maximise performance. Importantly, PBT discovers a schedule of hyperparameter settings rather than following the generally sub-optimal strategy of trying to find a single fixed set to use for the whole course of training. With just a small modification to a typical distributed hyperparameter training framework, our method allows robust and reliable training of models. We demonstrate the effectiveness of PBT on deep reinforcement learning problems, showing faster wall-clock convergence and higher final performance of agents by optimising over a suite of hyperparameters. In addition, we show the same method can be applied to supervised learning for machine translation, where PBT is used to maximise the BLEU score directly, and also to training of Generative Adversarial Networks to maximise the Inception score of generated images. In all cases PBT results in the automatic discovery of hyperparameter schedules and model selection which results in stable training and better final performance.
hub tools
citation-role summary
background 1
citation-polarity summary
fields
cs.AI 11 cs.LG 10 cs.RO 4 cs.CL 3 cs.NE 3 cs.CV 2 cond-mat.stat-mech 1 cs.CE 1 cs.MA 1 physics.chem-ph 1roles
background 1polarities
unclear 1representative citing papers
LLMZero uses LLM agents to search training trajectories and discovers that capacity parameters accumulate monotonically while regularization parameters oscillate, leading to performance improvements of 9-140% on GRPO tasks.
Presents a query-complexity framework for genetic algorithms with guided operators and shows necessity of multiple operators and tight bounds for diversity in solution pools.
HARBOR is a new agentic harness framework that automates robot RL workflows end-to-end across 16 tasks in manipulation, locomotion, and dexterous control, matching or exceeding default configurations while enabling sim-to-real transfer.
New benchmark ICRL4AHT reveals that history-conditioned ICRL methods fail to show robust adaptation in multi-agent Overcooked-V2, underperforming random baselines on unseen teammates and layouts.
Evolutionary coding agents achieve most benchmark gains through a small subset of edit types and by cycling previously deleted code lines rather than developing new algorithmic structures.
EvoPref applies NSGA-II evolutionary optimization with archive-based diversity to populations of LoRA adapters, yielding 18% higher preference coverage and 47% lower collapse than gradient descent baselines while matching alignment quality.
Experimental comparison of 15 HPO and NAS algorithms for automated feature preprocessing on 45 tabular datasets finds evolution-based methods and random search as top performers.
3D-ALP achieves 0.65 success on memory-dependent 5-step robotic reach tasks versus near-zero for reactive baselines by anchoring MCTS planning to a persistent 3D camera-to-world frame.
MetaPS trains models via simulation rollouts to select from programmatic strategy libraries for market agents, yielding better performance than fixed or direct LLM baselines across model sizes.
REINFORCE self-training on competitive programming tasks exhibits robust rise-then-collapse in pass@1; CARE, ES, and GRPO mitigate it in model-size-dependent ways across Qwen-2.5-3B/7B and a Gemma pilot.
Scaling population size during training of emergent sketching agents increases zero-shot mutual intelligibility between independent groups by raising in-group variation and driving perceptual grounding.
SAGE compares social co-evolution against matched self-evolution across three arenas and finds peer history enables breakthroughs only for agents that plateau under self-improvement, with abstraction of traces mattering more than raw volume.
MLIPilot deploys LLM agents to autonomously optimize MACE MLIP training on molecular and periodic datasets by proposing code edits and validating against a domain-specific scorecard.
PopuLoRA shows that co-evolving populations of LoRA adapters through cross-evaluated self-play can outperform compute-matched single-agent baselines on multiple code and math reasoning benchmarks.
IBTS framework uses influence shaping to improve zero-shot human-machine teaming beyond partner diversity alone, with gains shown in Overcooked-AI simulations and a 30-subject human study.
An LLM acting as real-time controller for SIMP topology optimization parameters outperforms fixed schedules and heuristics, delivering 5.7-18.1% lower compliance on 2D and 3D benchmarks.
DERL is a differentiable bi-level method that evolves optimal reward structures for RL policies by composing atomic primitives and using meta-gradients from validation performance.
PPD integrates PPO into policy distillation so the student collects and uses its own rewards, yielding better sample efficiency and robustness than standard student-distill or teacher-distill on ATARI, Mujoco, and Procgen tasks.
Attention mechanism dynamically groups task knowledge at state granularity in multi-task DRL to enable positive transfer and avoid negative transfer, matching or exceeding prior methods with fewer parameters.
RGSE adapts text embeddings at test time via evolutionary search, using cosine similarity rewards from high-confidence visual proposals to improve open-vocabulary object detection under distribution shifts.
Isaac Lab is a unified GPU-native platform combining high-fidelity physics, photorealistic rendering, multi-frequency sensors, domain randomization, and learning pipelines for scalable multi-modal robot policy training.
An effective reward function emerges that fully governs the evolution of policy distributions across the population, yielding closed equations for mean and variance under Gaussian assumptions.
An LLM-driven evolutionary framework generates executable trading strategies as Python code and uses a meta-loop to evolve the prompts that guide synthesis.
citing papers explorer
-
Promptbreeder: Self-Referential Self-Improvement Via Prompt Evolution
Promptbreeder evolves both task prompts and the mutation prompts that improve them using LLMs, outperforming Chain-of-Thought and Plan-and-Solve on arithmetic and commonsense reasoning benchmarks.
-
LLMZero: Discovering Adaptive Training Strategies for RL Post-Training via LLM Agents
LLMZero uses LLM agents to search training trajectories and discovers that capacity parameters accumulate monotonically while regularization parameters oscillate, leading to performance improvements of 9-140% on GRPO tasks.
-
Mathematical perspective on genetic algorithms with optimization guided operators
Presents a query-complexity framework for genetic algorithms with guided operators and shows necessity of multiple operators and tight bounds for diversity in solution pools.
-
HARBOR: A Harness Framework for Agentic Robot Reinforcement Learning
HARBOR is a new agentic harness framework that automates robot RL workflows end-to-end across 16 tasks in manipulation, locomotion, and dexterous control, matching or exceeding default configurations while enabling sim-to-real transfer.
-
Benchmarking the Limits of In-Context Reinforcement Learning for Ad-Hoc Teamwork
New benchmark ICRL4AHT reveals that history-conditioned ICRL methods fail to show robust adaptation in multi-agent Overcooked-V2, underperforming random baselines on unseen teammates and layouts.
-
What Do Evolutionary Coding Agents Evolve?
Evolutionary coding agents achieve most benchmark gains through a small subset of edit types and by cycling previously deleted code lines rather than developing new algorithmic structures.
-
EvoPref: Multi-Objective Evolutionary Optimization Discovers Diverse LLM Alignments Beyond Gradient Descent
EvoPref applies NSGA-II evolutionary optimization with archive-based diversity to populations of LoRA adapters, yielding 18% higher preference coverage and 47% lower collapse than gradient descent baselines while matching alignment quality.
-
Auto-FP: An Experimental Study of Automated Feature Preprocessing for Tabular Data
Experimental comparison of 15 HPO and NAS algorithms for automated feature preprocessing on 45 tabular datasets finds evolution-based methods and random search as top performers.
-
3D-Anchored Lookahead Planning for Persistent Robotic Scene Memory via World-Model-Based MCTS
3D-ALP achieves 0.65 success on memory-dependent 5-step robotic reach tasks versus near-zero for reactive baselines by anchoring MCTS planning to a persistent 3D camera-to-world frame.
-
MetaPS: Adaptive Programmatic Strategy Selection for Market Agents
MetaPS trains models via simulation rollouts to select from programmatic strategy libraries for market agents, yielding better performance than fixed or direct LLM baselines across model sizes.
-
Self-Improvement Can Self-Regress: The Rise-and-Collapse Failure Mode of LLM Self-Training
REINFORCE self-training on competitive programming tasks exhibits robust rise-then-collapse in pass@1; CARE, ES, and GRPO mitigate it in model-size-dependent ways across Qwen-2.5-3B/7B and a Gemma pilot.
-
Drawing with Strangers: Population Scaling Drives Zero-Shot Mutual Intelligibility in Emergent Sketching
Scaling population size during training of emergent sketching agents increases zero-shot mutual intelligibility between independent groups by raising in-group variation and driving perceptual grounding.
-
SAGE: A Quantitative Evaluation of Socialized Evolution in Agent Ecosystems
SAGE compares social co-evolution against matched self-evolution across three arenas and finds peer history enables breakthroughs only for agents that plateau under self-improvement, with abstraction of traces mattering more than raw volume.
-
MLIPilot: LLM-Driven Auto-Research for Machine-Learned Interatomic Potentials
MLIPilot deploys LLM agents to autonomously optimize MACE MLIP training on molecular and periodic datasets by proposing code edits and validating against a domain-specific scorecard.
-
PopuLoRA: Co-Evolving LLM Populations for Reasoning Self-Play
PopuLoRA shows that co-evolving populations of LoRA adapters through cross-evaluated self-play can outperform compute-matched single-agent baselines on multiple code and math reasoning benchmarks.
-
Beyond Partner Diversity: An Influence-Based Team Steering Framework for Zero-Shot Human-Machine Teaming
IBTS framework uses influence shaping to improve zero-shot human-machine teaming beyond partner diversity alone, with gains shown in Overcooked-AI simulations and a 30-subject human study.
-
Large Language Models as Optimization Controllers: Adaptive Continuation for SIMP Topology Optimization
An LLM acting as real-time controller for SIMP topology optimization parameters outperforms fixed schedules and heuristics, delivering 5.7-18.1% lower compliance on 2D and 3D benchmarks.
-
Differentiable Evolutionary Reinforcement Learning
DERL is a differentiable bi-level method that evolves optimal reward structures for RL policies by composing atomic primitives and using meta-gradients from validation performance.
-
Proximal Policy Distillation
PPD integrates PPO into policy distillation so the student collects and uses its own rewards, yielding better sample efficiency and robustness than standard student-distill or teacher-distill on ATARI, Mujoco, and Procgen tasks.
-
Attentive Multi-Task Deep Reinforcement Learning
Attention mechanism dynamically groups task knowledge at state granularity in multi-task DRL to enable positive transfer and avoid negative transfer, matching or exceeding prior methods with fewer parameters.
-
Reward-Guided Semantic Evolution for Test-time Adaptive Object Detection
RGSE adapts text embeddings at test time via evolutionary search, using cosine similarity rewards from high-confidence visual proposals to improve open-vocabulary object detection under distribution shifts.
-
Isaac Lab: A GPU-Accelerated Simulation Framework for Multi-Modal Robot Learning
Isaac Lab is a unified GPU-native platform combining high-fidelity physics, photorealistic rendering, multi-frequency sensors, domain randomization, and learning pipelines for scalable multi-modal robot policy training.
-
Theory of collective learning in populations of adaptive agents
An effective reward function emerges that fully governs the evolution of policy distributions across the population, yielding closed equations for mean and variance under Gaussian assumptions.
-
AlgoEvolve: LLM-driven Meta-evolution of Algorithmic Trading Programs
An LLM-driven evolutionary framework generates executable trading strategies as Python code and uses a meta-loop to evolve the prompts that guide synthesis.
-
FORGE: Self-Evolving Agent Memory With No Weight Updates via Population Broadcast
FORGE is a staged population protocol that evolves prompt-injected memory (Rules, Examples, or Mixed) for ReAct agents via reflection and broadcast, yielding 1.7-7.7× gains over zero-shot and 29-72% over Reflexion on CybORG CAGE-2.
-
Growing Action Spaces
A curriculum of growing action spaces combined with simultaneous off-policy value estimation accelerates learning in large multi-agent action spaces.
-
Can Tabular Foundation Models Guide Exploration in Robot Policy Learning?
TFM-S3 uses a tabular foundation model to predict returns and guide intermittent global exploration within an SVD-derived policy subspace, yielding faster early convergence and better final performance than TD3 and population-based methods under fixed rollout budgets.
-
Scalable Hyperparameter-Divergent Ensemble Training with Automatic Learning Rate Exploration for Large Models
HDET lets data-parallel replicas explore a spread of learning rates independently before averaging parameters, with an auto-LR controller driven by inter-replica loss differences to produce a self-adapting schedule without extra sweeps.
-
Generalization Guarantees on Data-Driven Tuning of Gradient Descent with Langevin Updates
LGD reaches Bayes optimality at optimal hyperparameters and admits an O(dh) pseudo-dimension bound for meta-learning hyperparameters on convex regression tasks.
-
Vocabulary Dropout for Curriculum Diversity in LLM Co-Evolution
Randomly masking a proposer's output vocabulary during training and generation sustains curriculum diversity and improves solver accuracy by +4.4 points at 8B in LLM co-evolution.
-
AI Training Manager: Bounded Closed-Loop Control of Adaptive Training Recipes
An LLM-based bounded controller adapts ML training parameters from structured telemetry to correct overfitting and exploration issues, shown on TinyStories and robotic RL tasks.
-
A Reinforcement Learning Inspired Latent Yield Based Adaptive Algorithm Switching Mechanism
Introduces a latent yield mechanism for stable adaptive algorithm switching using RL-inspired aggregation and GA island models, with experiments on sorting and robotic obstacle avoidance.
-
Acoustic Model Optimization Based On Evolutionary Stochastic Gradient Descent with Anchors for Automatic Speech Recognition
ESGD with anchors guarantees no degradation from the anchor model and reports improved loss and ASR performance on BN50 and SWB300 datasets.
-
EvoNash-MARL: A Closed-Loop Multi-Agent Reinforcement Learning Framework for Medium-Horizon Equity Allocation
EvoNash-MARL achieves 19.6% annualized returns on equity allocation from 2014-2024 versus 11.7% for SPY, with evidence of robustness under constraints but no strong statistical superiority per WRC and SPA-lite tests.
-
Staged Factorial Screening for Budget-Constrained Micro-Pretraining
Staged factorial screening recovers stable early penalties from total batch, depth, and width in 2-10 minute pretraining runs and supports a bridge-centered recommendation through 24-hour continuations on two hosts.
-
Automated Machine Learning in Practice: State of the Art and Recent Results
Survey of AutoML methods with benchmarks on their performance for business applications.
-
On the notion of number in humans and machines
Experiments indicate deep learning models achieve higher accuracy on numerosity tasks for counts below human subitizing capacity.