Pith. sign in

REVIEW 4 major objections 6 minor 129 references

Towards Universal Offline Black-Box Optimization via Learning Language Model Embeddings

T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper argues that a single string-based model, UniSO, can solve offline black-box optimization across heterogeneous design spaces and generalize to unseen tasks once its embedding space is shaped by metadata-guided contrastive…

desk verdict A clear, reproducible proof of concept for LM-based universal offline BBO, but the zero-shot claim leans on task-revealing metadata and the few-shot numbers smell like surrogate overfitting. read the letter →

arxiv 2506.07109 v1 pith:BTME6YTP submitted 2025-06-08 cs.LG cs.AIcs.NE

Rong-Xi Tan , Ming Chen , Ke Xue , Yao Wang , Yaoyuan Wang , Sheng Fu , Chao Qian This is my paper
classification cs.LGcs.AIcs.NE
keywords offlineblack-boxoptimizationuniversallanguagemodelembeddingsstring-basedrepresentationmulti-taskregressionmetadataconditioningcontrastivelearningzero-shotgeneralization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to establish that offline black-box optimization does not have to be rebuilt from scratch for every new problem. It proposes UniSO, a framework that writes each candidate design as a text string, prepends a short metadata note saying what task it is and what objective is being optimized, and trains one language-model-based regressor on all tasks together. The payoff would be a single optimizer that handles continuous, categorical, and high-dimensional spaces of different sizes, transfers knowledge between related tasks, and needs only a handful of examples to adapt to a task it has never seen. The paper reports that its improved token-targeted variant, UniSO-T, beats single-task numeric-input experts on several Design-Bench and SOO-Bench tasks, and that both variants generalize zero-shot and few-shot to unseen tasks. If the claim holds, it breaks the traditional fixed-dimensional, single-task barrier of offline BBO.

What carries the argument

The machinery that carries the argument is a task-shared string embedding space plus two geometric regularizers. Designs are tokenized as JSON-like strings, prefixed with metadata that names the task, describes it, and states the objective; the same SentencePiece tokenizer maps these strings to inputs for a T5-style encoder. UniSO-T uses an encoder-decoder that autoregressively predicts the P10-encoded score tokens, while UniSO-N fits an MLP regressor on mean-pooled LM embeddings. The decisive components are the two losses added to the main regression objective: a contrastive loss that encourages agreement between the input-embedding similarity matrix and the metadata-embedding similarity matrix, which separates dissimilar tasks while keeping related tasks close, and a per-task Lipschitz loss that penalizes pairs whose embedding distance is small while their score difference is large, which enforces local smoothness. A loss-balancing rule scales the auxiliary gradients by the ratio of the main loss to each auxiliary loss, so the regularizers steer rather than dominate training.

What would settle it

Train the improved UniSO-T on the same nine Design-Bench and SOO-Bench tasks, but replace every task's metadata with one fixed generic sentence such as "optimize the given design" while keeping all other hyperparameters and search settings identical. If the model then loses its edge over single-task numeric-input experts on the in-distribution tasks and its zero-shot advantage on RobotPush, Rover, and LunarLander, the claimed universality is attributable to metadata conditioning rather than to learning a general string-embedding space.

Watch

Extended reading notes

Core claim

The central claim is that a unified string-based representation, combined with a well-shaped learned embedding space, is sufficient for universal offline BBO. Concretely, the paper shows that a single multi-task regressor can score designs from heterogeneous search spaces when every design is serialized as a JSON-like dictionary and prefixed by hand-written metadata consisting of task name, description, and objective. It instantiates this as two variants: UniSO-T predicts the numerical score token-by-token using the P10 digit encoding inside a T5-based encoder-decoder, while UniSO-N embeds the input strings with an LM encoder and regresses the score with an MLP. The paper's improvements are two losses applied to the embedding space: a contrastive loss that aligns input embeddings with metadata embeddings so that similar tasks cluster together, and a Lipschitz loss that enforces local smoothness so that nearby embeddings correspond to similar scores. In its main experiments, improved UniSO-T outperforms single-task numeric-input experts on several tasks, and both variants exceed the best scores in the offline datasets and beat a z-score-normalized expert after few-shot fine-tuning on unseen tasks. The paper also argues, from attention visualizations and from training-from-scratch comparisons, that pre-trained language-model priors can be harmful for numeric regression because they concentrate attention on structural tokens such as EOS rather than on the numeric tokens that matter for optimization.

Load-bearing premise

The load-bearing premise is that the hand-written metadata supplied to the model, namely the task name, description, and objective, is a fair task description rather than a leak of the answer. The paper's own ablation shows that removing pieces of metadata degrades performance, so if generic metadata fails to preserve the gains, the universality claim collapses.

Editorial extensions

If this is right

  • If one universal regressor can replace per-task surrogates, practitioners no longer need to collect a large dataset for every new design problem; cross-task data can be pooled.
  • The zero-shot and few-shot results on RobotPush, Rover, and LunarLander suggest that a universal model can bootstrap a new task from only its worst 100 offline examples and already outperform a single-task expert trained on the same data.
  • Because the representation is string-based, the same model can in principle cover continuous, categorical, integer, and permutation variables without changing architecture.
  • The finding that pre-trained LM embeddings can hurt numeric regression implies that string-based BBO models should be trained from scratch or from checkpoints with strong mathematical content rather than defaulting to general-purpose LMs.
  • The model-inner search can use any off-the-shelf black-box optimizer over strings, including Bayesian optimization, evolutionary algorithms, and CMA-ES, so the universal surrogate plugs into existing search software.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the strongest untested premise is whether the hand-written metadata is a legitimate task descriptor or a leakage channel; a fair stress test would give generic metadata to every task and check whether the cross-task advantage survives.
  • Editorial inference: the attention analysis suggests that future universal optimizers could improve by adapting language models to arithmetic-heavy and code-heavy text, but the paper stops short of proving that such adaptation transfers to downstream optimization performance.
  • Editorial inference: the framework is demonstrated on single-objective unconstrained tasks, leaving open whether the same string-embedding recipe supports constrained, multi-objective, or mixed-integer search spaces common in real engineering and scientific design.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes UniSO, a universal offline black-box optimization framework that represents designs as strings, conditions on hand-written task metadata (name, description, objective), and learns a regressor whose latent space is regularized by a contrastive metadata-alignment loss and a Lipschitz smoothness loss. Two instantiations are studied: UniSO-T, a token-targeted T5-style sequence-to-sequence regressor, and UniSO-N, a numeric-targeted regressor with an MLP head on top of an embedding model. Experiments cover nine Design-Bench and SOO-Bench tasks for multi-task training, plus zero-shot and few-shot evaluation on RobotPush, Rover, and LunarLander. The paper reports that improved UniSO-T outperforms the authors' batch-normalized single-task MLP experts in several tasks, and that zero-/few-shot results on unseen tasks exceed the best scores in the offline datasets. The central claim is that unifying language-model priors with a learned string-embedding space can overcome traditional barriers in universal offline BBO.

Significance. If the universality claim were fully established, the work would be a meaningful step toward a single model that solves heterogeneous offline BBO tasks and transfers to new problems. The paper deserves credit for releasing code, providing ablation studies for each proposed component, and honestly reporting a lower rank (9.8/22) against published single-task offline BBO methods in Appendix E.1. The attention-visualization analysis of pre-trained versus from-scratch embedders is also a useful empirical observation. However, the evidence for the headline claim is thin: only nine training tasks, only three held-out tasks from the same control/simulation family, and no significance testing in a regime where standard deviations are very large. The strongest result about 'unseen-task generalization' is confounded by the fact that the model receives each unseen task's explicit optimization objective as metadata, as shown by the paper's own ablation in Table 14. The contribution is promising but the claims currently outrun the evidence.

major comments (4)
  1. [§3.3, §4.2 (RQ3), Appendix D Table 7, Appendix E.5 Table 14] The zero-shot generalization result is confounded by the metadata. For each unseen task, the metadata explicitly states the optimization objective (e.g., 'maximize the mean terminal reward' for LunarLander and 'minimize the total velocity change' for GTOPX in Table 7). Because the contrastive loss in §3.3 aligns the input-embedding similarity to the metadata-embedding similarity, the model can implement text-conditioned retrieval or conditioning on the stated objective rather than learning a genuinely transferable string-embedding representation for new tasks. The paper's own ablation in Table 14 shows that removing all metadata drops zero-shot LunarLander from 31.186 ± 27.971 to 6.251 ± 53.042, below D(best) = 7.038, while full metadata gives the inflated value. This is a load-bearing issue for the universality claim: the authors should present zero-shot results with generic or ablated metadata and explicitly separate 'knowing the task objective from text' from 'generalizing to a new task.'
  2. [§4.2 (RQ7), Appendix E.1 Table 8] The paper's central claim that UniSO 'achieves better results than single-task numeric-input experts' is based on the authors' own batch-normalized MLP baselines in Tables 1 and 2. Against 21 published single-task offline BBO methods using the field-standard z-score normalization protocol (Table 8), improved UniSO-T ranks 9.8 on average and does not outperform state-of-the-art methods. The abstract and conclusion should qualify the comparative claim to the specific weak baseline, and the inconsistency between the BN-based comparison (Tables 1 and 2) and the z-score-based comparison (Table 8) should be explicitly reconciled.
  3. [§4.2 (RQ1, RQ2), Tables 1 and 2] The performance comparisons lack statistical support. Many reported standard deviations are extremely large (e.g., Ant 241.350 ± 288.922 in Table 1; TF Bind 10 0.929 ± 0.802 in Table 2), and the overlapping confidence intervals across methods make it impossible to conclude that one method is better than another. The authors should report the number of seeds, paired differences, and significance tests (e.g., Wilcoxon signed-rank) for the claimed improvements, especially for the UniSO-T versus expert comparisons that are central to RQ1 and RQ2.
  4. [§4.2 (RQ3), Figure 4, Appendix C.3] The evidence for universality is thin. Zero-shot generalization is tested on only three tasks (RobotPush, Rover, LunarLander), all drawn from the same simulation/control suite of Wang et al. (2018; 2024a), and the training tasks are only nine Design-Bench/SOO-Bench problems. This does not support the broad claim of 'overcoming traditional barriers in universal BBO' across heterogeneous design spaces and task types. Additional held-out tasks from different families (e.g., molecule, protein, or mixed categorical-continuous problems) and an analysis of what transfers (embedding geometry vs. per-task retrieval) are needed before the universality claim can be accepted.
minor comments (6)
  1. [Appendix F, Figure 15 caption] The caption for Figure 15 says the plots are on the GTOPX 6 task, but the subfigures are labeled TF Bind 10; the caption should be corrected.
  2. [§2.1] The sentence 'where X could be be CONTINUOUS...' has a duplicated 'be' and is grammatically incomplete; it should read 'where X could be CONTINUOUS...'.
  3. [Appendix A.1] There is a typo: 'BO is is a widely used sample-efficient method' should be 'BO is a widely used sample-efficient method'.
  4. [Appendix A.3] The word 'tabluar' should be 'tabular' in the sentence about Transformers performing well on tabular data.
  5. [§4.2 (RQ4)] The text contains 'the rank correlation between between predicted and ground-truth objective scores'; the duplicated 'between' should be removed.
  6. [References] The Journal of Machine Learning Research reference for Raffel et al. is spelled 'Maching' instead of 'Machine'.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: UniSO is a supervised multi-task regression pipeline whose central claims are empirically tested against external benchmarks and the paper's own ablations.

full rationale

The paper's derivation chain is not circular. UniSO-T and UniSO-N are trained with stringified designs, task metadata, and objective labels through standard cross-entropy or MSE losses plus contrastive and Lipschitz regularizers; no predicted quantity is defined in terms of a fitted parameter. The zero-shot evaluation on RobotPush, Rover, and LunarLander uses hand-crafted metadata as a conditional input, and the paper explicitly acknowledges that metadata 'can ... hint the information of the unknown objective function f' (Sec. 2.1). This is a legitimate (if potentially leaky) conditioning variable in a text-conditioned regression model, and the paper transparently reports an ablation removing metadata (Table 14), which shows the effect is task-dependent rather than a by-construction identity. The self-citations, notably Tan et al. (2025) by overlapping authors, are used only as a comparison baseline, for benchmark-exclusion conventions, and for a related-work critique of regression; they are not invoked to prove the paper's central claim or to forbid alternatives. No uniqueness theorem is imported from the authors' prior work, and the P10/string representation and T5 embedder are adopted from external sources with the relevant assumptions stated. The main weakness of the paper is a potential task-information leak in the unseen-task experiments, which is a correctness/validity concern rather than circularity.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The central claim rests on a small number of domain assumptions: string tokenization preserves numeric structure, metadata is a legitimate task prior, and the two regularization losses shape the embedding space as intended. No new physical or mathematical entities are introduced.

free parameters (2)
  • temperature tau in contrastive loss = not specified in paper
    Hyperparameter of the contrastive embedding alignment loss in Section 3.3; chosen by the authors and not reported.
  • Lipschitz constant L = median of pairwise Lipschitz ratios
    Set to the median of |yi-yj|/||zi-zj|| over the task dataset (Section 3.4), so it is data-derived rather than fit to the final results, but the median choice is a manual heuristic.
assumptions (4)
  • domain assumption String representation of designs via JSON-like dictionaries preserves the optimization-relevant structure across heterogeneous search spaces.
    Used throughout Section 3.1.1; if tokenization destroys numerical relationships, the surrogate model cannot learn.
  • domain assumption Hand-crafted metadata (name, description, objective) provides a sufficient and non-circular signal for task discrimination and transfer.
    Section 3.1.2 and zero-shot experiments (Section 4.2 RQ3); the metadata description states the objective directly, which may leak test-task information.
  • domain assumption Contrastive alignment of design embeddings with metadata embeddings improves optimization by separating tasks in latent space, as evidenced by t-SNE plots.
    Section 3.3; relies on the visual interpretation of t-SNE as a proxy for embedding geometry.
  • domain assumption The Lipschitz regularization with L set as the median of pairwise ratios stably enforces local smoothness without distorting the embedding space.
    Section 3.4; the choice of median is heuristic, following Lee et al. 2023.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Towards Universal Offline Black-Box Optimization via Learning Language Model Embeddings." pith.science (2026). https://pith.science/paper/BTME6YTP

@misc{pith2026250607109,
  author       = {Pith},
  title        = {Pith review of: Towards Universal Offline Black-Box Optimization via Learning Language Model Embeddings},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BTME6YTP}},
  note         = {Machine review of arXiv:2506.07109}
}
read the original abstract

The pursuit of universal black-box optimization (BBO) algorithms is a longstanding goal. However, unlike domains such as language or vision, where scaling structured data has driven generalization, progress in offline BBO remains hindered by the lack of unified representations for heterogeneous numerical spaces. Thus, existing offline BBO approaches are constrained to single-task and fixed-dimensional settings, failing to achieve cross-domain universal optimization. Recent advances in language models (LMs) offer a promising path forward: their embeddings capture latent relationships in a unifying way, enabling universal optimization across different data types possible. In this paper, we discuss multiple potential approaches, including an end-to-end learning framework in the form of next-token prediction, as well as prioritizing the learning of latent spaces with strong representational capabilities. To validate the effectiveness of these methods, we collect offline BBO tasks and data from open-source academic works for training. Experiments demonstrate the universality and effectiveness of our proposed methods. Our findings suggest that unifying language model priors and learning string embedding space can overcome traditional barriers in universal BBO, paving the way for general-purpose BBO algorithms. The code is provided at https://github.com/lamda-bbo/universal-offline-bbo.

Figures

Figures reproduced from arXiv: 2506.07109 by the authors.

Figure 1
Figure 1. Framework of universal string-based offline BBO. 3. Method In this section, we introduce potential methods to solve universal offline BBO. We begin by outlining the general framework of universal string-based offline BBO in Sec￾tion 3.1, where we will present two variants based on mod￾eling. Then, we discuss the issues of these variants in Sec￾tion 3.2, which motivate us to improve the framework via metadata guidanc… view at source ↗
Figure 3
Figure 3. t-SNE plots comparing embedding distributions of vanilla UniSO-T and improved UniSO-T on five Design￾Bench (Trabucco et al., 2022) tasks. The embedding distribu￾tions of vanilla UniSO-T show mixed and overlapping embeddings with a circular pattern, lacking clear task boundaries. Our im￾proved method (right) achieves three improvements: (1) separating embeddings into distinct clusters for better task discrimination, … view at source ↗
Figure 4
Figure 4. Normalized performance comparison between zero-shot inference and few-shot finetuning of UniSO-T (left) and UniSO-N (right) across three tasks (RobotPush, Rover, and LunarLander). The light blue bars represent zero-shot performance, the dark blue bars show the improved performance after few-shot finetuning. The red dashed line denotes the best score D(best) in the few-shot offline dataset. 7 [PITH_FULL_IMAGE:figure… view at source ↗
Figures from the paper (23 more)
Figure 5
Figure 5. Figure 5: Left: Normalized performance comparison between UniSO-N trained from scratch (blue) and UniSO-N with pre￾trained embedder (yellow) across four tasks in SOO-Bench. The red dashed line denotes the best score D(best) in the offline dataset. Right: Comparison of OOD Spearm…
Figure 6
Figure 6. Figure 6: Attention weight distribution comparison of different models on the TF Bind 10 (Barrera et al., 2016) task: pre-trained T5-Small embedder (top left), T5-Small embedder trained from scratch (top right), pre-trained Qwen2.5-1.5B (bottom left), and pre-trained DeepSeek-R1…
Figure 7
Figure 7. Figure 7: Time distribution comparison between algorithm time and model inference time for different UniSO models (UniSO-N, UniSO￾T) and optimization methods (BO-qEI, EA). The pie charts show the percentage breakdown between algorithm time (purple) and inference time (orange) fo…
Figure 8
Figure 8. Figure 8: Training loss curves comparing UniSO-N with T5-Small versus T5-Base embedders during training. E.5. Ablation Studies on Each Component To thoroughly evaluate the effectiveness of each component our method, we conduct ablation studies on metadata quality, loss component…
Figure 9
Figure 9. Figure 9: t-SNE plots of UniSO-N embedder comparing BO-qEI (left) and EA (right) search trajectories. Each part represents data from a single task, and small scatters represent the search trajectories of these two optimizers. Different colors represent different tasks, with dark…
Figure 10
Figure 10. Figure 10 [PITH_FULL_IMAGE:figures/full_fig_p029_10.png]
Figure 11
Figure 11. Figure 11: Performance comparison of improved UniSO-N combined with different EoH (Liu et al., 2024a) strategies. The bar plot shows the normalized performance of the baseline UniSO-N model and three EoH-enhanced variants (EoH-M: metadata summarization, EoH-R: regularization-bas…
Figure 12
Figure 12. Figure 12: Average attention distribution of pre-trained T5-small embedder on data from different tasks. The plots present attention patterns for 12 distinct tasks: Ant, D’Kitty, Superconductor, TF Bind 8, TF Bind 10, GTOPX 2, GTOPX 3 , GTOPX 4 , GTOPX 6, RobotPush, Rover, and L…
Figure 13
Figure 13. Figure 13: Average attention distribution of T5-small embedder trained from scratch on different tasks. The plots show attention patterns across various components for 12 different tasks: Ant, D’Kitty, Superconductor, TF Bind 8, TF Bind 10, GTOPX 2, GTOPX 3, GTOPX 4, GTOPX 6, Ro…
Figure 14
Figure 14. Figure 14: Different language models’ average attention distribution on GTOPX 6 task. The bar plots compare the normalized attention scores across different components for four models: T5-Small (top left), Qwen2.5-1.5B (top right), T5-Base (bottom left), and DeepSeek￾R1-Distil-Q…
Figure 15
Figure 15. Figure 15: Different language models’ average attention distribution on GTOPX 6 task. The bar plots compare the normalized attention scores across different components for four models: T5-Small (top left), Qwen2.5-1.5B (top right), T5-Base (bottom left), and DeepSeek￾R1-Distil-Q…
Figure 16
Figure 16. Figure 16: Two examples of prompt engineering used in initialization and E2 strategy for EoH-M. 35 [PITH_FULL_IMAGE:figures/full_fig_p035_16.png]
Figure 17
Figure 17. Figure 17: Two examples of prompt engineering used in E1 and M1 strategy for EoH-M. 36 [PITH_FULL_IMAGE:figures/full_fig_p036_17.png]
Figure 18
Figure 18. Figure 18: Two examples of prompt engineering used in M3 strategy for EoH-M. Note that we do not have an M2 strategy, since M2 in EoH (Liu et al., 2024a) represents hyper-parameter adaptation while metadata do not have any hyper-parameters. 37 [PITH_FULL_IMAGE:figures/full_fig_…
Figure 19
Figure 19. Figure 19: Two examples of prompt engineering used in initialization and E2 strategy for EoH-R. 38 [PITH_FULL_IMAGE:figures/full_fig_p038_19.png]
Figure 20
Figure 20. Figure 20: Two examples of prompt engineering used in E1 and M1 strategy for EoH-R. 39 [PITH_FULL_IMAGE:figures/full_fig_p039_20.png]
Figure 21
Figure 21. Figure 21: Two examples of prompt engineering used in M2 and M3 strategy for EoH-R. 40 [PITH_FULL_IMAGE:figures/full_fig_p040_21.png]
Figure 22
Figure 22. Figure 22: Two examples of prompt engineering used in initialization and E2 strategy for EoH-A. 41 [PITH_FULL_IMAGE:figures/full_fig_p041_22.png]
Figure 23
Figure 23. Figure 23: Two examples of prompt engineering used in E1 and M1 strategy for EoH-A. 42 [PITH_FULL_IMAGE:figures/full_fig_p042_23.png]
Figure 24
Figure 24. Figure 24: Two examples of prompt engineering used in M2 and M3 strategy for EoH-A. 43 [PITH_FULL_IMAGE:figures/full_fig_p043_24.png]
Figure 25
Figure 25. Figure 25: Best metadata found by EoH-M. 44 [PITH_FULL_IMAGE:figures/full_fig_p044_25.png]
Figure 26
Figure 26. Figure 26: Best code found by EoH-R. Description by the model: the algorithm extends the local structure preservation concept by incorporating adaptive kernel-based density estimation with multi-scale diffusion distances, while utilizing task-specific repulsion forces and smooth…
Figure 27
Figure 27. Figure 27: Best code of EoH-A. Description by the model: balance pairwise distances in both embedding and metadata spaces while enforcing consistency through rank correlation. 46 [PITH_FULL_IMAGE:figures/full_fig_p046_27.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

129 extracted references · 62 canonical work pages

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    ROBEL : R obotics benchmarks for learning with low-cost robots

    Ahn, M., Zhu, H., Hartikainen, K., Ponte, H., Gupta, A., Levine, S., and Kumar, V. ROBEL : R obotics benchmarks for learning with low-cost robots. In Proceedings of the 4th Conference on Robot Learning (CoRL), pp.\ 1300--1313, Virtual, 2020

    2020

  3. [3]

    and Li, Y

    Allen-Zhu , Z. and Li, Y. Physics of language models: P art 1, learning hierarchical language structures. arXiv:2305.13673, 2023

    arXiv 2023

  4. [4]

    The Claude 3 model family: Opus, Sonnet, Haiku

    Anthropic. The Claude 3 model family: Opus, Sonnet, Haiku . Anthropic AI Technical Report, 2024

    2024

  5. [5]

    Evolutionary Algorithms in Theory and Practice: Evolution Strategies, Evolutionary Programming, Genetic Algorithms

    B\" a ck, T. Evolutionary Algorithms in Theory and Practice: Evolution Strategies, Evolutionary Programming, Genetic Algorithms . Oxford University Press, 1996

    1996

  6. [6]

    Transfer learning for B ayesian optimization: A survey

    Bai, T., Li, Y., Shen, Y., Zhang, X., Zhang, W., and Cui, B. Transfer learning for B ayesian optimization: A survey. arXiv:2302.05927, 2023

    arXiv 2023

  7. [7]

    R., Daulton, S., Letham, B., Wilson, A

    Balandat, M., Karrer, B., Jiang, D. R., Daulton, S., Letham, B., Wilson, A. G., and Bakshy, E. Botorch: A framework for efficient Monte-Carlo B ayesian optimization. In Advances in Neural Information Processing Systems 33 (NeurIPS), pp.\ 21524--21538, Virtual, 2020

    2020

  8. [8]

    A., Vedenko, A., Kurland, J

    Barrera, L. A., Vedenko, A., Kurland, J. V., Rogers, J. M., Gisselbrecht, S. S., Rossin, E. J., Woodard, J. C., Mariani, L., Kock, K. H., Inukai, S., Siggers, T., Shokri, L., Gord \^a n, R., Sahni, N., Cotsapas, C., Hao, T., Yi, S. S., Kellis, M., Daly, M. J., Vidal, M., Hill, D. E., and Bulyk, M. L. Survey of variation in human transcription factors reve...

    2016

Show all 129 references
  1. [9]

    and Deb , K

    Blank , J. and Deb , K. pymoo: M ulti-objective optimization in P ython. IEEE Access, 8: 0 89497--89509, 2020

    2020

  2. [10]

    Open AI G ym

    Brockman, G., Cheung, V., Pettersson, L., Schneider, J., Schulman, J., Tang, J., and Zaremba, W. Open AI G ym. arXiv:1606.01540, 2016

    2016 arXiv

  3. [11]

    Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D. M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., L...

    1901

  4. [12]

    Linear algebra with T ransformers

    Charton, F. Linear algebra with T ransformers. Transactions on Machine Learning Research, 2022

    2022

  5. [13]

    N., and Doppa, J

    Chemingui, Y., Deshwal, A., Hoang, T. N., and Doppa, J. R. Offline model-based optimization via policy-guided gradient search. In Proceedings of the 38th AAAI Conference on Artificial Intelligence (AAAI), pp.\ 11230--11239, Vancouver, Canada, 2024

    2024

  6. [14]

    Robust guided diffusion for offline black-box optimization

    Chen, C., Beckham, C., Liu, Z., Liu, X., and Pal, C. Robust guided diffusion for offline black-box optimization. Transactions on Machine Learning Research, 2024

    2024

  7. [15]

    S., Zhang, Y., Fu, J., Liu, X

    Chen, C. S., Zhang, Y., Fu, J., Liu, X. S., and Coates, M. Bidirectional learning for offline infinite-width model-based optimization. In Advances in Neural Information Processing Systems 36 (NeurIPS), pp.\ 29454--29467, New Orleans, LA, 2022 a

    2022

  8. [16]

    S., Beckham, C., Liu, Z., Liu, X

    Chen, C. S., Beckham, C., Liu, Z., Liu, X. S., and Pal, C. Parallel-mentoring for offline model-based optimization. In Advances in Neural Information Processing Systems 37 (NeurIPS), pp.\ 76619--76636, New Orleans, LA, 2023 a

    2023

  9. [17]

    S., Zhang, Y., Liu, X

    Chen, C. S., Zhang, Y., Liu, X. S., and Coates, M. Bidirectional learning for offline model-based biological sequence design. In Proceedings of the 40th International Conference on Machine Learning (ICML), pp.\ 5351--5366, Honolulu, HI, 2023 b

    2023

  10. [18]

    A simple framework for contrastive learning of visual representations

    Chen, T., Kornblith, S., Norouzi, M., and Hinton, G. A simple framework for contrastive learning of visual representations. In Proceedings of the 37th International Conference on Machine Learning (ICML), pp.\ 1597--1607, Virtual, 2020

    2020

  11. [19]

    Towards learning universal hyperparameter optimizers with T ransformers

    Chen, Y., Song, X., Lee, C., Wang, Z., Zhang, Q., Dohan, D., Kawakami, K., Kochanski, G., Doucet, A., Ranzato, M., Perel, S., and de Freitas, N. Towards learning universal hyperparameter optimizers with T ransformers. In Advances in Neural Information Processing Systems 36 (Ne...

    2022

  12. [20]

    G rad N orm: G radient normalization for adaptive loss balancing in deep multitask networks

    Chen, Z., Badrinarayanan, V., Lee, C.-Y., and Rabinovich, A. G rad N orm: G radient normalization for adaptive loss balancing in deep multitask networks. In Proceedings of the 35th International Conference on Machine Learning (ICML), pp.\ 794--803, Stockholm, Sweden, 2018

    2018

  13. [21]

    C., Nguyen, P

    Dao, M. C., Nguyen, P. L., Truong, T. N., and Hoang, T. N. Incorporating surrogate gradient norm to improve offline optimization techniques. In Advances in Neural Information Processing Systems 38 (NeurIPS), pp.\ 8014--8046, Vancouver, Canada, 2024 a

    2024

  14. [22]

    C., Nguyen, P

    Dao, M. C., Nguyen, P. L., Truong, T. N., and Hoang, T. N. Boosting offline optimizers with surrogate sensitivity. In Proceedings of the 41st International Conference on Machine Learning (ICML), pp.\ 10072--10090, Vienna, Austria, 2024 b

    2024

  15. [23]

    S., Babu, C

    Dara, S., Dhamercherla, S., Jadav, S. S., Babu, C. M., and Ahsan, M. J. Machine learning in drug discovery: A review. Artificial Intelligence Review, 55 0 (3): 0 1947--1999, 2022

    1947

  16. [24]

    DeepSeek-R1 : I ncentivizing reasoning capability in LLMs via reinforcement learning

    DeepSeek-AI, Guo, D., Yang, D., Zhang, H., Song, J., Zhang, R., Xu, R., Zhu, Q., Ma, S., Wang, P., Bi, X., et al. DeepSeek-R1 : I ncentivizing reasoning capability in LLMs via reinforcement learning. arXiv:2501.12948, 2025

    2025 arXiv

  17. [25]

    M., Friesen, A

    Dery, L. M., Friesen, A. L., Freitas, N. D., Ranzato, M., and Chen, Y. Multi-step planning for automated hyperparameter optimization with OptFormer . In Foundation Models for Decision Making Workshop at NeurIPS'22, New Orleans, LA, 2022

    2022

  18. [26]

    Transfer learning for B ayesian optimization on heterogeneous search spaces

    Fan, Z., Han, X., and Wang, Z. Transfer learning for B ayesian optimization on heterogeneous search spaces. Transactions on Machine Learning Research, 2024

    2024

  19. [27]

    and Listgarten, J

    Fannjiang, C. and Listgarten, J. Autofocused oracles for model-based design. In Advances in Neural Information Processing Systems 33 (NeurIPS), pp.\ 12945--12956, Virtual, 2020

    2020

  20. [28]

    Frazier, P. I. A tutorial on B ayesian optimization. arXiv:1807.02811, 2018

    2018 arXiv

  21. [29]

    and Levine, S

    Fu, J. and Levine, S. Offline model-based optimization via normalized maximum likelihood estimation. In Proceedings of the 9th International Conference on Learning Representations (ICLR), Virtual, 2021

    2021

  22. [30]

    Metadata conditioning accelerates language model pre-training

    Gao, T., Wettig, A., He, L., Dong, Y., Malladi, S., and Chen, D. Metadata conditioning accelerates language model pre-training. arXiv:2501.01956, 2025

    2025 arXiv

  23. [31]

    S., and Valiant, G

    Garg, S., Tsipras, D., Liang, P. S., and Valiant, G. What can T ransformers learn in-context? A case study of simple function classes. In Advances in Neural Information Processing Systems 36 (NeurIPS), pp.\ 30583--30598, New Orleans, LA, 2022

    2022

  24. [32]

    W., Rezende, D., and Eslami, S

    Garnelo, M., Rosenbaum, D., Maddison, C., Ramalho, T., Saxton, D., Shanahan, M., Teh, Y. W., Rezende, D., and Eslami, S. A. Conditional neural processes. In Proceedings of the 34th International Conference on Machine Learning (ICML), pp.\ 1704--1713, Stockholm, Sweden, 2018

    2018

  25. [33]

    B ayesian Optimization

    Garnett, R. B ayesian Optimization . Cambridge University Press, 2023

    2023

  26. [34]

    A., and Hennig, P

    Garnett, R., Osborne, M. A., and Hennig, P. Active learning of linear embeddings for G aussian processes. In Proceedings of the 30th Conference on Uncertainty in Artificial Intelligence (UAI), pp.\ 230--239, Quebec, Canada, 2014

    2014

  27. [35]

    J., Bento, A

    Gaulton, A., Bellis, L. J., Bento, A. P., Chambers, J., Davies, M., Hersey, A., Light, Y., McGlinchey, S., Michalovich, D., Al-Lazikani, B., and Overington, J. P. Ch EMBL : A large-scale bioactivity database for drug discovery. Nucleic A cids R esearch , 40 0 (D1): 0 D1100--D1...

    2012

  28. [36]

    S.-H., Shi, Y.-H., and Zhang, J

    Gong, Y.-J., Li, J.-J., Zhou, Y., Li, Y., Chung, H. S.-H., Shi, Y.-H., and Zhang, J. Genetic learning particle swarm optimization. IEEE Transactions on Cybernetics, 46 0 (10): 0 2277--2290, 2015

    2015

  29. [37]

    Generative adversarial networks

    Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., and Bengio, Y. Generative adversarial networks. In Advances in Neural Information Processing Systems 27 (NeurIPS), pp.\ 139--144, Montreal, Canada, 2014

    2014

  30. [38]

    Soft actor-critic algorithms and applications

    Haarnoja, T., Zhou, A., Hartikainen, K., Tucker, G., Ha, S., Tan, J., Kumar, V., Zhu, H., Gupta, A., Abbeel, P., and Levine, S. Soft actor-critic algorithms and applications. arXiv:1812.05905, 2018

    2018 arXiv

  31. [39]

    A data-driven statistical model for predicting the critical temperature of a superconductor

    Hamidieh, K. A data-driven statistical model for predicting the critical temperature of a superconductor. Computational M aterials S cience , 154: 0 346--354, 2018

    2018

  32. [40]

    The CMA evolution strategy: A tutorial

    Hansen, N. The CMA evolution strategy: A tutorial. arXiv:1604.00772, 2016

    2016 arXiv

  33. [41]

    V., and Auger, A

    Hansen, N., Arnold, D. V., and Auger, A. Evolution strategies. In Springer Handbook of Computational Intelligence, pp.\ 871--898. Springer, 2015

    2015

  34. [42]

    MetaBalance : I mproving multi-task recommendations via adapting gradient magnitudes of auxiliary tasks

    He, Y., Feng, X., Cheng, C., Ji, G., Guo, Y., and Caverlee, J. MetaBalance : I mproving multi-task recommendations via adapting gradient magnitudes of auxiliary tasks. In Proceedings of the ACM Web Conference 2022 (WWW), pp.\ 2205--2215, Lyon, France, 2022

    2022

  35. [43]

    E., Srivastava, N., Krizhevsky, A., Sutskever, I., and Salakhutdinov, R

    Hinton, G. E., Srivastava, N., Krizhevsky, A., Sutskever, I., and Salakhutdinov, R. Improving neural networks by preventing co-adaptation of feature detectors. arXiv:1207.0580, 2012

    2012 arXiv

  36. [44]

    Denoising diffusion probabilistic models

    Ho, J., Jain, A., and Abbeel, P. Denoising diffusion probabilistic models. In Advances in Neural Information Processing Systems 33 (NeurIPS), pp.\ 6840--6851, Virtual, 2020

    2020

  37. [45]

    Hoang, M., Fadhel, A., Deshwal, A., Doppa, J., and Hoang, T. N. Learning surrogates for offline black-box optimization via gradient matching. In Proceedings of the 41st International Conference on Machine Learning (ICML), pp.\ 18374--18393, Vienna, Austria, 2024

    2024

  38. [46]

    u ller, S., Purucker, L., Krishnakumar, A., K \

    Hollmann, N., M \"u ller, S., Purucker, L., Krishnakumar, A., K \"o rfer, M., Hoo, S. B., Schirrmeister, R. T., and Hutter, F. Accurate predictions on small data with a tabular foundation model. Nature, 637 0 (8045): 0 319--326, 2025

    2025

  39. [47]

    BO : A ugmenting acquisition functions with user beliefs for B ayesian optimization

    Hvarfner, C., Stoll, D., Souza, A., Nardi, L., Lindauer, M., and Hutter, F. BO : A ugmenting acquisition functions with user beliefs for B ayesian optimization. In Proceedings of the 10th International Conference on Learning Representations (ICLR), Virtual, 2022

    2022

  40. [48]

    A general framework for user-guided B ayesian optimization

    Hvarfner, C., Hutter, F., and Nardi, L. A general framework for user-guided B ayesian optimization. In Proceedings of the 12th International Conference on Learning Representations (ICLR), Vienna, Austria, 2024

    2024

  41. [49]

    Global Optimization and Space Pruning for Spacecraft Trajectory Design , pp.\ 178--199

    Izzo, D. Global Optimization and Space Pruning for Spacecraft Trajectory Design , pp.\ 178--199. Cambridge University Press, 2010

    2010

  42. [50]

    and Manuel L \' o pez-Ib \' a \ n ez, M

    Izzo, D. and Manuel L \' o pez-Ib \' a \ n ez, M. Optimization challenges at the European Space Agency . In Proceedings of the 24th ACM Genetic and Evolutionary Computation Conference (GECCO), pp.\ 1542--1553, Boston, MA, 2022

    2022

  43. [51]

    LLMOPT : L earning to define and solve general optimization problems from scratch

    Jiang, C., Shu, X., Qian, H., Lu, X., Zhou, J., Zhou, A., and Yu, Y. LLMOPT : L earning to define and solve general optimization problems from scratch. In Proceedings of the 13th International Conference on Learning Representations (ICLR), Singapore, 2025

    2025

  44. [52]

    and Eberhart, R

    Kennedy, J. and Eberhart, R. Particle swarm optimization. In Proceedings of International Conference on Neural Networks (ICNN), pp.\ 1942--1948, Perth, Australia, 1995

    1942

  45. [53]

    I., Grosnit, A., Deik, D.-G.-X., Robert, P

    Khan, A., Cowen-Rivers, A. I., Grosnit, A., Deik, D.-G.-X., Robert, P. A., Greiff, V., Smorodina, E., Rawat, P., Dreczkowski, K., Akbar, R., Tutunov, R., Bou-Ammar, D., Wang, J., Storkey, A., and Bou-Ammar, H. Toward real-world automated antibody design with combinatorial B ay...

    2023

  46. [54]

    Bootstrapped training of score-conditioned generator for offline design of biological sequences

    Kim, M., Berto, F., Ahn, S., and Park, J. Bootstrapped training of score-conditioned generator for offline design of biological sequences. In Advances in Neural Information Processing Systems 36 (NeurIPS), pp.\ 67643--67661, New Orleans, LA, 2023

    2023

  47. [55]

    Offline model-based optimization: C omprehensive review

    Kim, M., Gu, J., Yuan, Y., Yun, T., Liu, Z., Bengio, Y., and Chen, C. Offline model-based optimization: C omprehensive review. arXiv:2503.17286, 2025

    2025

  48. [56]

    D., Wu, D., Wang, H., Ferber, A., Ma, Y.-A., Gomes, C

    Kong, L., Du, Y., Mu, W., Neklyudov, K., Bortoli, V. D., Wu, D., Wang, H., Ferber, A., Ma, Y.-A., Gomes, C. P., and Zhang, C. Diffusion models as constrained samplers for optimization with unknown constraints. In Generative Models for Decision Making workshop at ICLR'24, Vienn...

    2024

  49. [57]

    M., and Grover, A

    Krishnamoorthy, S., Mashkaria, S. M., and Grover, A. Diffusion models for black-box optimization. In Proceedings of the 40th International Conference on Machine Learning (ICML), pp.\ 17842--17857, Honolulu, HI, 2023

    2023

  50. [58]

    G., Abbeel, P., and Levine, S

    Kuba, J. G., Abbeel, P., and Levine, S. Cliqueformer : M odel-based optimization with structured T ransformers. arXiv:2410.13106, 2024 a

    2024

  51. [59]

    G., Uehara, M., Levine, S., and Abbeel, P

    Kuba, J. G., Uehara, M., Levine, S., and Abbeel, P. Functional graphical models: Structure enables offline data-driven optimization. In Proceedings of the 27th International Conference on Artificial Intelligence and Statistics (AISTATS), pp.\ 2449--2457, Valencia, Spain, 2024 b

    2024

  52. [60]

    and Richardson, J

    Kudo, T. and Richardson, J. S entence P iece: A simple and language independent subword tokenizer and detokenizer for neural text processing. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing: System Demonstrations (EMNLP), pp.\ 66--71, ...

    2018

  53. [61]

    and Levine, S

    Kumar, A. and Levine, S. Model inversion networks for model-based optimization. In Advances in Neural Information Processing Systems 33 (NeurIPS), pp.\ 5126--5137, Virtual, 2020

    2020

  54. [62]

    Data-driven offline optimization for architecting hardware accelerators

    Kumar, A., Yazdanbakhsh, A., Hashemi, M., Swersky, K., and Levine, S. Data-driven offline optimization for architecting hardware accelerators. In Proceedings of the 10th International Conference on Learning Representations (ICLR), Virtual, 2022

    2022

  55. [63]

    Lange, R. T. evosax: JAX -based evolution strategies. In Proceedings of the 25th Companion Conference on Genetic and Evolutionary Computation (GECCO), pp.\ 659--662, Lisbon, Portugal, 2023

    2023

  56. [64]

    Lee, S., Chu, J., Kim, S., Ko, J., and Kim, H. J. Advancing B ayesian optimization via learning correlated latent space. In Advances in Neural Information Processing Systems 37 (NeurIPS), pp.\ 48906--48917, New Orleans, LA, 2023

    2023

  57. [65]

    Lehre, P. K. and Lin, S. No free lunch theorem and black-box complexity analysis for adversarial optimisation. In Advances in Neural Information Processing Systems 38 (NeurIPS), pp.\ 121570--121597, Vancouver, Canada, 2024

    2024

  58. [66]

    The power of scale for parameter-efficient prompt tuning

    Lester, B., Al-Rfou, R., and Constant, N. The power of scale for parameter-efficient prompt tuning. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp.\ 3045--3059, Punta Cana, Dominican Republic, 2021

    2021

  59. [67]

    C., Hutter, F., Feurer, M., and Bischl, B

    Lindauer, M., Karl, F., Klier, A., Moosbauer, J., Tornede, A., Mueller, A. C., Hutter, F., Feurer, M., and Bischl, B. Position: A call to action for a human-centered A uto ML paradigm. In Proceedings of the 41st International Conference on Machine Learning (ICML), pp.\ 30566--...

    2024

  60. [68]

    Evolution of heuristics: T owards efficient automatic algorithm design using large language model

    Liu, F., Xialiang, T., Yuan, M., Lin, X., Luo, F., Wang, Z., Lu, Z., and Zhang, Q. Evolution of heuristics: T owards efficient automatic algorithm design using large language model. In Proceedings of the 41st International Conference on Machine Learning (ICML), pp.\ 32201--322...

    2024

  61. [69]

    A systematic survey on large language models for algorithm design

    Liu, F., Yao, Y., Guo, P., Yang, Z., Zhao, Z., Lin, X., Tong, X., Yuan, M., Lu, Z., Wang, Z., et al. A systematic survey on large language models for algorithm design. arXiv:2410.14716, 2024 b

    2024

  62. [70]

    LLM4AD : A platform for algorithm design with large language model

    Liu, F., Zhang, R., Xie, Z., Sun, R., Li, K., Lin, X., Wang, Z., Lu, Z., and Zhang, Q. LLM4AD : A platform for algorithm design with large language model. arXiv:2412.17287, 2024 c

    2024

  63. [71]

    J., Saleh, M., Pot, E., Goodrich, B., Sepassi, R., Kaiser, L., and Shazeer, N

    Liu, P. J., Saleh, M., Pot, E., Goodrich, B., Sepassi, R., Kaiser, L., and Shazeer, N. Generating Wikipedia by summarizing long sequences. In Proceedings of the 6th International Conference on Learning Representations (ICLR), Vancouver, Canada, 2018

    2018

  64. [72]

    Large language models to enhance B ayesian optimization

    Liu, T., Astorga, N., Seedat, N., and van der Schaar, M. Large language models to enhance B ayesian optimization. In Proceedings of the 12th International Conference on Learning Representations (ICLR), Vienna, Austria, 2024 d

    2024

  65. [73]

    and Hutter, F

    Loshchilov, I. and Hutter, F. Decoupled weight decay regularization. In Proceedings of the 7th International Conference on Learning Representations (ICLR), New Orleans, LA, 2019

    2019

  66. [74]

    Degradation-resistant offline optimization via accumulative risk control

    Lu, H., Qian, H., Wu, Y., Liu, Z., Zhang, Y., Zhou, A., and Yu, Y. Degradation-resistant offline optimization via accumulative risk control. In Proceedings of the 26th European Conference on Artificial Intelligence (ECAI), pp.\ 1609--1616, Krak \'o w, Poland, 2023

    2023

  67. [75]

    LLaMoCo : I nstruction tuning of large language models for optimization code generation

    Ma, Z., Guo, H., Chen, J., Peng, G., Cao, Z., Ma, Y., and Gong, Y.-J. LLaMoCo : I nstruction tuning of large language models for optimization code generation. arXiv:2403.01131, 2024

    2024 arXiv

  68. [76]

    Maaten, L. V. and Hinton, G. Visualizing data using t-SNE . Journal of Machine Learning Research, 9 0 (86): 0 2579--2605, 2008

    2008

  69. [77]

    Maraval, A., Zimmer, M., Grosnit, A., and Ammar, H. B. End-to-end meta- B ayesian optimisation with T ransformer neural processes. In Advances in Neural Information Processing Systems 37 (NeurIPS), pp.\ 11246--11260, New Orleans, LA, 2023

    2023

  70. [78]

    M., Krishnamoorthy, S., and Grover, A

    Mashkaria, S. M., Krishnamoorthy, S., and Grover, A. Generative pretraining for black-box optimization. In Proceedings of the 40th International Conference on Machine Learning (ICML), pp.\ 24173--24197, Honolulu, HI, 2023

    2023

  71. [79]

    P., Grabocka, J., and Hutter, F

    M \"u ller, S., Hollmann, N., Arango, S. P., Grabocka, J., and Hutter, F. Transformers can do B ayesian inference. In Proceedings of the 10th International Conference on Learning Representations (ICLR), Virtual, 2022

    2022

  72. [80]

    PFNs4BO : I n-context learning for B ayesian optimization

    M\" u ller, S., Feurer, M., Hollmann, N., and Hutter, F. PFNs4BO : I n-context learning for B ayesian optimization. In Proceedings of the 40th International Conference on Machine Learning (ICML), pp.\ 25444--25470, Honolulu, HI, 2023

    2023

  73. [81]

    L., Zhang, F., and Barak, B

    Nakkiran, P., Kaplun, G., Kalimeris, D., Yang, T., Edelman, B. L., Zhang, F., and Barak, B. SGD on neural networks learns functions of increasing complexity. In Advances in Neural Information Processing Systems 33 (NeurIPS), pp.\ 3496--3506, Vancouver, Canada, 2019

    2019

  74. [82]

    and Grover, A

    Nguyen, T. and Grover, A. Transformer neural processes: U ncertainty-aware meta learning via sequence modeling. In Proceedings of the 39th International Conference on Machine Learning (ICML), pp.\ 16569--16594, Baltimore, MD, 2022

    2022

  75. [83]

    and Grover, A

    Nguyen, T. and Grover, A. LICO : L arge language models for in-context molecular optimization. In Proceedings of the 13th International Conference on Learning Representations (ICLR), Singapore, 2025

    2025

  76. [84]

    ExPT : S ynthetic pretraining for few-shot experimental design

    Nguyen, T., Agrawal, S., and Grover, A. ExPT : S ynthetic pretraining for few-shot experimental design. In Advances in Neural Information Processing Systems 36 (NeurIPS), pp.\ 45856--45869, New Orleans, LA, 2023

    2023

  77. [85]

    Predicting from strings: L anguage model embeddings for B ayesian optimization

    Nguyen, T., Zhang, Q., Yang, B., Lee, C., Bornschein, J., Perel, S., Chen, Y., and Song, X. Predicting from strings: L anguage model embeddings for B ayesian optimization. arXiv:2410.10190, 2024

    2024

  78. [86]

    Z., Shirobokov, S., Kozlovskii, B., Ruiz, F

    Novikov, A., V \ u , N., Eisenberger, M., Dupont, E., Huang, P.-S., Wagner, A. Z., Shirobokov, S., Kozlovskii, B., Ruiz, F. J. R., Mehrabian, A., Kumar, M. P., See, A., Chaudhuri, S., Holland, G., Davies, A., Nowozin, S., Kohli, P., and Balog, M. AlphaEvolve: A coding agent fo...

    2025

  79. [87]

    Introduction to Game Physics with Box2D

    Parberry, I. Introduction to Game Physics with Box2D. CRC Press, 2017

    2017

  80. [88]

    Data-driven offline decision-making via invariant representation learning

    Qi, H., Su, Y., Kumar, A., and Levine, S. Data-driven offline decision-making via invariant representation learning. In Advances in Neural Information Processing Systems 36 (NeurIPS), pp.\ 13226--13237, New Orleans, LA, 2022

    2022

  81. [89]

    SOO - B ench: B enchmarks for evaluating the stability of offline black-box optimization

    Qian, H., Zhu, Y., Shu, X., An, X., Wen, Y., Liu, S., Lu, H., Zhou, A., Tang, K., and Yu, Y. SOO - B ench: B enchmarks for evaluating the stability of offline black-box optimization. In Proceedings of the 13th International Conference on Learning Representations (ICLR), Singap...

    2025

  82. [90]

    Qwen2.5 technical report

    Qwen Team , Yang, A., Yang, B., Zhang, B., Hui, B., Zheng, B., Yu, B., Li, C., Liu, D., Huang, F., Wei, H., Lin, H., Yang, J., Tu, J., Zhang, J., Yang, J., Yang, J., Zhou, J., Lin, J., Dang, K., Lu, K., Bao, K., Yang, K., Yu, L., Li, M., Xue, M., Zhang, P., Zhu, Q., Men, R., L...

    2024 arXiv

  83. [91]

    Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., and Liu, P. J. Exploring the limits of transfer learning with a unified text-to-text T ransformer. Journal of Maching Learning Research, 21: 0 5485--5551, 2020

    2020

  84. [92]

    Rasmussen, C. E. and Williams, C. K. I. G aussian P rocesses for M achine L earning . The MIT Press, 2006

    2006

  85. [93]

    P., Dupont, E., Ruiz, F

    Romera-Paredes, B., Barekatain, M., Novikov, A., Balog, M., Kumar, M. P., Dupont, E., Ruiz, F. J. R., Ellenberg, J., Wang, P., Fawzi, O., Kohli, P., and Fawzi, A. Mathematical discoveries from program search with large language models. Nature, 625 0 (7995): 0 468--475, 2024

    2024

  86. [94]

    GTOPX space mission benchmarks

    Schlueter, M., Neshat, M., Wahib, M., Munetomo, M., and Wagner, M. GTOPX space mission benchmarks. SoftwareX, 14: 0 100666, 2021

    2021

  87. [95]

    P., and de Freitas, N

    Shahriari, B., Swersky, K., Wang, Z., Adams, R. P., and de Freitas, N. Taking the human out of the loop: A review of B ayesian optimization. Proceedings of the IEEE, 104 0 (1): 0 148--175, 2016

    2016

  88. [96]

    B., and Reddy, C

    Shojaee, P., Meidani, K., Gupta, S., Farimani, A. B., and Reddy, C. K. LLM-SR : S cientific equation discovery via programming with large language models. In Proceedings of the 13th International Conference on Learning Representation (ICLR), Singapore, 2025

    2025

  89. [97]

    Reinforced in-context black-box optimization

    Song, L., Gao, C., Xue, K., Wu, C., Li, D., Hao, J., Zhang, Z., and Qian, C. Reinforced in-context black-box optimization. In Proceedings of the 34th International Joint Conference on Artificial Intelligence (IJCAI), Montreal, Canada, 2025

    2025

  90. [98]

    and Bahri, D

    Song, X. and Bahri, D. Decoding-based regression. arXiv:2501.19383, 2025

    2025 arXiv

  91. [99]

    OmniPred : L anguage models as universal regressors

    Song, X., Li, O., Lee, C., Yang, B., Peng, D., Perel, S., and Chen, Y. OmniPred : L anguage models as universal regressors. Transactions on Machine Learning Research, 2024 a

    2024

  92. [100]

    T., Lee, C., Tang, Y., and Chen, Y

    Song, X., Tian, Y., Lange, R. T., Lee, C., Tang, Y., and Chen, Y. Position: L everage foundational models for black-box optimization. In Proceedings of the 41st International Conference on Machine Learning (ICML), pp.\ 46168--46180, Vienna, Austria, 2024 b

    2024

  93. [101]

    The V izier G aussian process bandit algorithm

    Song, X., Zhang, Q., Lee, C., Fertig, E., Huang, T.-K., Belenki, L., Kochanski, G., Ariafar, S., Vasudevan, S., Perel, S., and Golovin, D. The V izier G aussian process bandit algorithm. arXiv: 2408.11527, 2024 c

    2024 arXiv

  94. [102]

    J., Gruver, N., Maffettone, P., Delaney, E., Greenside, P., and Wilson, A

    Stanton, S., Maddox, W. J., Gruver, N., Maffettone, P., Delaney, E., Greenside, P., and Wilson, A. G. Accelerating B ayesian optimization for biological sequence design with denoising autoencoders. In Proceedings of the 39th International Conference on Machine Learning (ICML),...

    2022

  95. [103]

    Offline model-based optimization by learning to rank

    Tan, R.-X., Xue, K., Lyu, S.-H., Shang, H., Wang, Y., Wang, Y., Fu, S., and Qian, C. Offline model-based optimization by learning to rank. In Proceedings of the 13th International Conference on Learning Representations (ICLR), Singapore, 2025

    2025

  96. [104]

    and Ishibuchi, H

    Tanabe, R. and Ishibuchi, H. An easy-to-use real-world multi-objective optimization problem suite. Applied Soft Computing, 89: 0 106078, 2020

    2020

  97. [105]

    Understanding LLM embeddings for regression

    Tang, E., Yang, B., and Song, X. Understanding LLM embeddings for regression. Transactions on Machine Learning Research, 2025

    2025

  98. [106]

    Mu J o C o: A physics engine for model-based control

    Todorov, E., Erez, T., and Tassa, Y. Mu J o C o: A physics engine for model-based control. In 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp.\ 5026--5033, Algarve, Portugal, 2012

    2012

  99. [107]

    Conservative objective models for effective offline model-based optimization

    Trabucco, B., Kumar, A., Geng, X., and Levine, S. Conservative objective models for effective offline model-based optimization. In Proceedings of the 38th International Conference on Machine Learning (ICML), pp.\ 10358--10368, Virtual, 2021

    2021

  100. [108]

    D esign- B ench: B enchmarks for data-driven offline model-based optimization

    Trabucco, B., Geng, X., Kumar, A., and Levine, S. D esign- B ench: B enchmarks for data-driven offline model-based optimization. In Proceedings of the 39th International Conference on Machine Learning (ICML), pp.\ 21658--21676, Baltimore, MD, 2022

    2022

  101. [109]

    Bayesian optimization is superior to random search for machine learning hyperparameter tuning: Analysis of the black-box optimization challenge 2020

    Turner, R., Eriksson, D., McCourt, M., Kiili, J., Laaksonen, E., Xu, Z., and Guyon, I. Bayesian optimization is superior to random search for machine learning hyperparameter tuning: Analysis of the black-box optimization challenge 2020. In NeurIPS 2020 Competition and Demonstr...

    2020

  102. [110]

    and Van Der Schaar, M

    Van Breugel, B. and Van Der Schaar, M. Position: W hy tabular foundation models should be a research priority. In Proceedings of the 41st International Conference on Machine Learning (ICML), pp.\ 48976--48993, Vienna, Austria, 2024

    2024

  103. [111]

    Attention is all you need

    Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A., Kaiser, L., and Polosukhin, I. Attention is all you need. In Advances in Neural Information Processing Systems 31 (NeurIPS), pp.\ 6000--6010, Long Beach, CA, 2017

    2017

  104. [112]

    M onte C arlo tree search based space transfer for black box optimization

    Wang, S., Xue, K., Song, L., Huang, X., and Qian, C. M onte C arlo tree search based space transfer for black box optimization. In Advances in Neural Information Processing Systems 38 (NeurIPS), pp.\ 49591--49624, Vancouver, Canada, 2024 a

    2024

  105. [113]

    Batched large-scale B ayesian optimization in high-dimensional spaces

    Wang, Z., Gehring, C., Kohli, P., and Jegelka, S. Batched large-scale B ayesian optimization in high-dimensional spaces. In Proceedings of the 21st International Conference on Artificial Intelligence and Statistics (AISTATS), pp.\ 745--754, Lanzarote, Spain, 2018

    2018

  106. [114]

    E., Swersky, K., Lee, C., Nado, Z., Gilmer, J., Snoek, J., and Ghahramani, Z

    Wang, Z., Dahl, G. E., Swersky, K., Lee, C., Nado, Z., Gilmer, J., Snoek, J., and Ghahramani, Z. Pre-trained G aussian processes for B ayesian optimization. Journal of Machine Learning Research, 25 0 (212): 0 1--83, 2024 b

    2024

  107. [115]

    Scalable B ayesian optimization via focalized sparse G aussian processes

    Wei, Y., Zhuang, V., Soedarmadji, S., and Sui, Y. Scalable B ayesian optimization via focalized sparse G aussian processes. In Advances in Neural Information Processing Systems 38 (NeurIPS), pp.\ 120443--120467, Vancouver, Canada,, 2024

    2024

  108. [116]

    Evaluating the robustness of neural networks: A n extreme value theory approach

    Weng, T., Zhang, H., Chen, P., Yi, J., Su, D., Gao, Y., Hsieh, C., and Daniel, L. Evaluating the robustness of neural networks: A n extreme value theory approach. In Proceedings of the 6th International Conference on Learning Representations (ICLR), Vancouver, Canada, 2018

    2018

  109. [117]

    and Grabocka, J

    Wistuba, M. and Grabocka, J. Few-shot B ayesian optimization with deep kernel surrogates. In Proceedings of the 9th International Conference on Learning Representations (ICLR), Virtual, 2021

    2021

  110. [118]

    and Macready, W

    Wolpert, D. and Macready, W. No free lunch theorems for optimization. IEEE Transactions on Evolutionary Computation, 1 0 (1): 0 67--82, 1997

    1997

  111. [119]

    Offline multi-objective optimization

    Xue, K., Tan, R.-X., Huang, X., and Qian, C. Offline multi-objective optimization. In Proceedings of the 41st International Conference on Machine Learning (ICML), pp.\ 55595--55624, Vienna, Austria, 2024

    2024

  112. [120]

    S., Zeng, Y., Bastani, H., Gardner, J., Gee, J

    Yao, M. S., Zeng, Y., Bastani, H., Gardner, J., Gee, J. C., and Bastani, O. Generative adversarial model-based optimization via source critic regularization. In Advances in Neural Information Processing Systems 38 (NeurIPS), pp.\ 44009--44039, Vancouver, Canada, 2024

    2024

  113. [121]

    Multi-objective evolution of heuristic using large language model

    Yao, S., Liu, F., Lin, X., Lu, Z., Wang, Z., and Zhang, Q. Multi-objective evolution of heuristic using large language model. In Proceedings of the 39th AAAI Conference on Artificial Intelligence (AAAI), pp.\ 27144--27152, Philadelphia, PA, 2025

    2025

  114. [122]

    Yu, P., Zhang, D., He, H., Ma, X., Miao, R., Lu, Y., Zhang, Y., Kong, D., Gao, R., Xie, J., Cheng, G., and Wu, Y. N. Latent energy-based O dyssey: B lack-box optimization via expanded exploration in the energy-based latent space. arXiv:2405.16730, 2024

    2024 arXiv

  115. [123]

    Ro MA : R obust model adaptation for offline model-based optimization

    Yu, S., Ahn, S., Song, L., and Shin, J. Ro MA : R obust model adaptation for offline model-based optimization. In Advances in Neural Information Processing Systems 34 (NeurIPS), pp.\ 4619--4631, Virtual, 2021

    2021

  116. [124]

    S., Liu, Z., Neiswanger, W., and Liu, X

    Yuan, Y., Chen, C. S., Liu, Z., Neiswanger, W., and Liu, X. S. Importance-aware co-teaching for offline model-based optimization. In Advances in Neural Information Processing Systems 37 (NeurIPS), pp.\ 55718--55733, New Orleans, LA, 2023

    2023

  117. [125]

    S., Wu, H., Li, Z., Li, J., Clark, J

    Yuan, Y., Zhang, Y., Chen, C. S., Wu, H., Li, Z., Li, J., Clark, J. J., and Liu, X. S. Design editing for offline model-based optimization. Transactions on Machine Learning Research, 2025

    2025

  118. [126]

    Guided trajectory generation with diffusion models for offline model-based optimization

    Yun, T., Yun, S., Lee, J., and Park, J. Guided trajectory generation with diffusion models for offline model-based optimization. In Advances in Neural Information Processing Systems 38 (NeurIPS), pp.\ 83847--83876, Vancouver, Canada, 2024

    2024

  119. [127]

    Design of physical experiments via collision-free latent space optimization

    Zhang, F., Altas, Y., Fan, L., Vinchure, K., Nord, B., and Chen, Y. Design of physical experiments via collision-free latent space optimization. In Machine Learning and the Physical Sciences Workshop at NeurIPS'20, Virtual, 2020

    2020

  120. [128]

    Monte C arlo tree search for comprehensive exploration in LLM -based automatic heuristic design

    Zheng, Z., Xie, Z., Wang, Z., and Hooi, B. Monte C arlo tree search for comprehensive exploration in LLM -based automatic heuristic design. In Proceedings of the 42nd International Conference on Machine Learning (ICML), Vancouver, Canada, 2025

    2025

  121. [129]

    Evolutionary Learning: Advances in Theories and Algorithms

    Zhou, Z.-H., Yu, Y., and Qian, C. Evolutionary Learning: Advances in Theories and Algorithms. Springer, 2019

    2019

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.