REVIEW 12 cited by
LLaMoCo: Instruction Tuning of Large Language Models for Optimization Code Generation
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
LLaMoCo: Instruction Tuning of Large Language Models for Optimization Code Generation
read the original abstract
Recent research explores optimization using large language models (LLMs) by either iteratively seeking next-step solutions from LLMs or directly prompting LLMs for an optimizer. However, these approaches exhibit inherent limitations, including low operational efficiency, high sensitivity to prompt design, and a lack of domain-specific knowledge. We introduce LLaMoCo, the first instruction-tuning framework designed to adapt LLMs for solving optimization problems in a code-to-code manner. Specifically, we establish a comprehensive instruction set containing well-described problem prompts and effective optimization codes. We then develop a novel two-phase learning strategy that incorporates a contrastive learning-based warm-up procedure before the instruction-tuning phase to enhance the convergence behavior during model fine-tuning. The experiment results demonstrate that a CodeGen (350M) model fine-tuned by our LLaMoCo achieves superior optimization performance compared to GPT-4 Turbo and the other competitors across both synthetic and realistic problem sets. The fine-tuned model and the usage instructions are available at https://anonymous.4open.science/r/LLaMoCo-722A.
Forward citations
Cited by 12 Pith papers
-
Meta-Black-Box Optimization Can Do Search Guidance for Expensive Constrained Multi-Objective Optimization
MetaSG-SAEA is a bi-level meta-BBO framework that uses a meta-policy for search guidance via the MM-CCI constraint abstraction and diffusion-based population initialization to outperform baselines on expensive constra...
-
ORPilot: A Production-Oriented Agentic LLM-for-OR Tool for Optimization Modeling
ORPilot is the first agentic LLM system built specifically for production optimization modeling, using interview, data collection, parameter computation agents and a solver-agnostic intermediate representation to hand...
-
PerfCoder: Large Language Models for Interpretable Code Performance Optimization
PerfCoder is a family of LLMs trained on optimization trajectories with human annotations and runtime-based preference alignment that achieves higher runtime speedups and optimization rates on the PIE benchmark than p...
-
Search Hardness-Aware LLM-Based Problem Formulation for Expensive Simulation-Driven Design
SHA-PF uses initial simulation data to select a 'hard but promising' anchor satisfaction state and evolves LLM-generated formulations that prioritize it, reaching target designs with fewer expensive simulations on ant...
-
MiniOpt: Reasoning to Model and Solve General Optimization Problems with Limited Resources
MiniOpt is an RL framework that decomposes optimization reasoning into modeling and solver generation, achieving top solving accuracy for models under 10B parameters across diverse problem types using OptReward and op...
-
ERFSL: An Efficient Reward Function Searcher via Language Models for Custom-Environment Multi-Objective Optimization (Student Abstract)
ERFSL generates and optimizes LLM-based reward functions for custom multi-objective RL, correcting codes in one iteration and converging weights in 5.2 iterations on average even from 500x errors.
-
Solver-Independent Automated Problem Formulation via LLMs for High-Cost Simulation-Driven Design
APF automates solver-independent formulation of optimization problems from natural language via LLMs fine-tuned on synthetically generated high-quality data, outperforming prior methods on antenna radiation efficiency tasks.
-
Language Models as Efficient Reward Function Searchers for Custom-Environment Multi-Objective Reinforcement
ERFSL uses LLMs to create per-requirement reward components, correct their code via a critic, and optimize weights with genetic-algorithm-style mutation and crossover driven by training logs, succeeding in a zero-shot...
-
MiniOpt: Reasoning to Model and Solve General Optimization Problems with Limited Resources
MiniOpt trains LLMs under 10B parameters via RL with OptReward to model and solve general optimization problems, reporting highest average solving accuracy among comparable models.
-
Vehicle Routing Problem Meets Large Language Models: An Overview and Perspectives
Survey organizing LLM uses for VRP into modeler, designer, and coordinator roles, covering variants, solvers, benchmarks, and two experiments.
-
Reward-Free Code Alignment from Pretrained or Fine-Tuned LLM: Unpacking the Trade-offs for Code Generation
Empirical study on five LLMs finds pretrained-to-aligned paths yield bigger gains over baseline than finetuned-to-aligned paths, though absolute accuracy remains lower for pretrained starts.
-
A Systematic Survey on Large Language Models for Evolutionary Optimization: From Modeling to Solving
A literature survey that classifies LLM-based optimization research into modeling and solving, with solving divided into LLMs as optimizers, low-level components, and high-level managers.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.