Pith. sign in

REVIEW 7 cited by

Using Large Language Models for Hyperparameter Optimization

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2312.04528 v2 pith:J5UITM2Z submitted 2023-12-07 cs.LG cs.AI

Michael R. Zhang , Nishkrit Desai , Juhan Bae , Jonathan Lorraine , Jimmy Ba This is my paper
classification cs.LGcs.AI
keywords hyperparameterllmsmodelsoptimizationmodelapproachesbenchmarkslanguage
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

This paper explores the use of foundational large language models (LLMs) in hyperparameter optimization (HPO). Hyperparameters are critical in determining the effectiveness of machine learning models, yet their optimization often relies on manual approaches in limited-budget settings. By prompting LLMs with dataset and model descriptions, we develop a methodology where LLMs suggest hyperparameter configurations, which are iteratively refined based on model performance. Our empirical evaluations on standard benchmarks reveal that within constrained search budgets, LLMs can match or outperform traditional HPO methods like Bayesian optimization across different models on standard benchmarks. Furthermore, we propose to treat the code specifying our model as a hyperparameter, which the LLM outputs and affords greater flexibility than existing HPO approaches.

Discussion (0). Sign in to comment.

Forward citations

Cited by 7 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Agentic Bayesian Optimization through Surrogate-Augmented Autoresearch

    cs.LG 2026-07 conditional novelty 6.0 of 10

    An LLM agent that fully controls a reconfigurable Bayesian-optimization backend preserves standard BO reliability, outperforms LLM-only optimizers, and exploits natural-language priors and mid-run problem reformulation.

  2. AgentHPOBench: A Benchmark For Evaluating LLM Agents as Sequential Hyperparameter Optimizers

    cs.AI 2026-07 conditional novelty 6.0 of 10

    A new 30-task benchmark shows LLM agents can improve real ML experiments through sequential hyperparameter choices, but their gains are uneven and often not retained.

  3. Which Optimizer, At What Budget? A Tournament of Optimizers for Search-Based SE

    cs.SE 2026-07 conditional novelty 6.0 of 10

    Optimizer choice in search-based SE is budget-dependent (EZR when labels are scarce, DE when plentiful), and a zero-probe table on objective structure and input-space shape predicts the winner ~75% of the time.

  4. A Language-Guided Bayesian Optimization for Efficient LoRA Hyperparameter Search

    cs.CL 2026-01 conditional novelty 5.0 of 10

    LLM embeddings plus Bayesian optimization find better LoRA hyperparameters in ~30 proxy trials than standard published settings.

  5. REMoH: A Reflective Evolution of Multi-objective Heuristics approach via Large Language Models

    cs.AI 2025-06 reject novelty 5.0 of 10

    REMoH evolves LLM-written heuristics with NSGA-II and a reflection mechanism, reporting competitive FJSSP results that are weakened by test-set selection.

  6. The Problem of Dynamic Spatial Sampling and Geofence Surveillance

    stat.AP 2026-03 unverdicted novelty 4.0 of 10

    Adaptive geofence radius estimators are proposed to trade off police reverse-location surveillance reach against local privacy under density-aware constraints.

  7. Language-Guided Tuning: Enhancing Numeric Optimization with Textual Feedback

    cs.AI 2025-08 reject novelty 4.0 of 10

    A multi-agent LLM framework uses natural-language 'textual gradients' to jointly tune architecture, features, training strategy, and hyperparameters.

Pith tools