REVIEW 11 cited by
ORLM: A Customizable Framework in Training Large Models for Automated Optimization Modeling
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
ORLM: A Customizable Framework in Training Large Models for Automated Optimization Modeling
read the original abstract
Optimization modeling plays a critical role in the application of Operations Research (OR) tools to address real-world problems, yet they pose challenges and require extensive expertise from OR experts. With the advent of large language models (LLMs), new opportunities have emerged to streamline and automate such task. However, current research predominantly relies on closed-source LLMs such as GPT-4, along with extensive prompt engineering techniques. This reliance stems from the scarcity of high-quality training datasets for optimization modeling, resulting in elevated costs, prolonged processing times, and privacy concerns. To address these challenges, our work is the first to propose a viable path for training open-source LLMs that are capable of optimization modeling and developing solver codes, eventually leading to a superior ability for automating optimization modeling and solving. Particularly, we design the {\sc OR-Instruct}, a semi-automated data synthesis framework for optimization modeling that enables customizable enhancements for specific scenarios or model types. This work also introduces IndustryOR, the first industrial benchmark for evaluating LLMs in solving practical OR problems. We train several 7B-scale open-source LLMs using synthesized data (dubbed ORLMs{https://github.com/Cardinal-Operations/ORLM}), which exhibit significantly enhanced optimization modeling capabilities, achieving competitive performance across the NL4OPT, MAMO, and IndustryOR benchmarks. Additionally, our experiments highlight the potential of scaling law and reinforcement learning to further enhance the performance of ORLMs. The workflows and human-machine interaction paradigms of ORLMs in practical industrial applications are also discussed in the paper.
Forward citations
Cited by 11 Pith papers
-
VeriSimpl: Robust Optimization Modeling from Natural Language using Simplification-based Verification
VeriSimpl improves natural-language-to-optimization translation by using the solver to generate simplified feasibility and variable-inference queries that an LLM checks against the problem text.
-
ORPilot: A Production-Oriented Agentic LLM-for-OR Tool for Optimization Modeling
ORPilot is the first agentic LLM system built specifically for production optimization modeling, using interview, data collection, parameter computation agents and a solver-agnostic intermediate representation to hand...
-
EngiBench: A Benchmark for Evaluating Large Language Models on Engineering Problem Solving
EngiBench shows LLMs accuracy drops with task complexity, degrades under perturbations, and stays below human performance on open-ended engineering problems.
-
YUKTI: From Natural-Language Situations to Robust, Verifiable Decisions An Uncertainty-Typed Proposition IR, Assumption-Robust Pareto Frontiers, and a Regret Certificate
An uncertainty-typed proposition IR plus Assumption-Robust Pareto Frontiers (ARPF) with a regret certificate cuts held-out regret by >90% under misspecification and beats status-quo and naive rules on real marketing data.
-
Learn2Zinc: Fine-tuning Small Language Models for Text-to-Model Translation in MiniZinc
Fine-tuning small LMs on synthetic and bootstrapped syntax-error corrections lifts MiniZinc execution accuracy from ~0% to 98% in an ensemble, but solution accuracy saturates near 35%.
-
PARM: Pipeline-Adapted Reward Model
PARM adapts reward models to multi-stage LLM pipelines via pipeline data and direct preference optimization, improving execution rate and solving accuracy on optimization benchmarks and showing transfer to GSM8K.
-
SAC-Opt: Semantic Anchors for Iterative Correction in Optimization Modeling
A semantic-anchor alignment loop that reconstructs and verifies constraints from generated solver code lifts average modeling accuracy by 7.8 percentage points over seven datasets.
-
Cooperative Design Optimization through Natural Language Interaction
An LLM-guided Bayesian optimization framework enables designers to intervene in design optimization via natural language, improving agency and reducing cognitive load without sacrificing performance.
-
LLM-Enhanced Multi-Agent Reinforcement Learning with Expert Workflow for Real-Time P2P Energy Trading
An LLM-enhanced MARL system with differential attention critic produces lower economic costs and voltage violations than baselines in simulated real-time P2P electricity trading.
-
Optimization and Constraint Modeling using LLMs with a Retrieval Augmented Generation Process
A RAG pipeline with 500 synthetic problem-solution pairs is claimed to improve LLM optimization modeling accuracy, but the evaluation compares different error tolerances between conditions.
-
Large Language Models in Transportation Systems Management and Operations: From Text Reasoning to Multi-modal Decision Support
A survey synthesizing LLM and MM-LLM uses in transportation operations, mobility services, and decision support while noting challenges like data heterogeneity and real-time needs.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.