COOPA: A Modular LLM Agent Architecture for Operations Research Problems

Chuanhao Li; Dirk Bergemann; Ethan X. Fang; Xiaoan Xu; Yehua Wei; Zhuoran Yang

arxiv: 2606.27611 · v1 · pith:JGRK4CNE · submitted 2026-06-25 · cs.LG

COOPA: A Modular LLM Agent Architecture for Operations Research Problems

Chuanhao Li , Xiaoan Xu , Dirk Bergemann , Ethan X. Fang , Yehua Wei , Zhuoran Yang This is my paper

Reviewed by Pith T0 review T1 audit T2 compute T3 formal T4 kernel 2026-06-29 01:09 UTCgrok-4.3pith:JGRK4CNE record.json open to challenge →

classification cs.LG

keywords LLM agentsoperations researchmodular architectureinterpretabilitymulti-solver routingconfidence modelingprovenancebenchmark evaluation

0 comments

The pith

COOPA modular architecture raises LLM accuracy on operations research tasks by up to 6.7 points over baselines.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper presents COOPA as a modular LLM-agent system that automates operations research modeling with three linked components. It claims this design produces higher accuracy and greater transparency than prior LLM approaches when evaluated on the same benchmarks and models. A sympathetic reader would care because OR problems involve high-stakes decisions that currently demand scarce human expertise in formulation and solver use. The reported results show the gains hold across eight different LLM backbones and four comparison methods under matched conditions.

Core claim

COOPA achieves the best macro-average accuracy on six of eight LLM backbones and improves over the strongest baseline by up to 6.7 percentage points across three OR benchmarks. The gains are produced by iterative confidence-based modeling that generates candidate formulations, scores them on multiple dimensions, and picks one via a max-min rule; element-level provenance that ties each variable, constraint, and objective to quoted source text; and multi-solver routing that dispatches to specialized optimizer agents for different problem classes. An ablation isolates the contribution of the iterative modeling step, while case studies illustrate the audit value of the provenance links.

What carries the argument

The COOPA architecture, which integrates iterative confidence-based modeling, element-level provenance and confidence explanations, and multi-solver routing to specialized optimizer agents.

If this is right

Iterative confidence-based modeling produces measurable accuracy gains, as shown by the within-system ablation.
Element-level provenance supplies a verifiable audit trail that links model outputs directly to source text.
Multi-solver routing enables handling of distinct OR problem classes through dispatch to specialized agents.
The accuracy advantages appear across eight LLM backbones when all methods are tested under identical conditions.
The full system outperforms four prior baselines in macro-average accuracy on the three benchmarks.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

The modular separation of modeling, explanation, and solving steps could be reused in other structured decision domains that mix formulation and optimization.
Source-linked explanations might lower the cost of human oversight in regulated or high-stakes modeling applications.
Extending the router to additional solver types could further widen coverage without changing the core agent loop.

Load-bearing premise

The three chosen benchmarks and four baselines fairly represent the space of OR problems and that performance differences are caused by the proposed components rather than unstated implementation choices or prompt engineering.

What would settle it

Running COOPA and the four baselines on a new, independently selected OR benchmark outside the original three and finding no accuracy advantage for COOPA would falsify the claim of general improvement.

Figures

Figures reproduced from arXiv: 2606.27611 by Chuanhao Li, Dirk Bergemann, Ethan X. Fang, Xiaoan Xu, Yehua Wei, Zhuoran Yang.

**Figure 2.** Figure 2: Per-benchmark ablation: iterative modeling ( [PITH_FULL_IMAGE:figures/full_fig_p032_2.png] view at source ↗

**Figure 3.** Figure 3: Confidence analysis of iterative refinement (aggregated across 3 datasets). (a) Min [PITH_FULL_IMAGE:figures/full_fig_p033_3.png] view at source ↗

**Figure 4.** Figure 4: Confidence gain vs. accuracy gain per model–benchmark pair. Each point represents [PITH_FULL_IMAGE:figures/full_fig_p034_4.png] view at source ↗

read the original abstract

Operations Research (OR) provides a rigorous framework for high-stakes decision-making, but effective OR modeling requires substantial domain knowledge, mathematical abstraction, and solver expertise. Recent LLM-based systems automate parts of this pipeline, yet remain limited by low accuracy on complex problems, opaque outputs, and narrow solver support. We propose COOPA (COoperative OPerations Agent), a modular LLM-agent architecture for interpretable and scalable OR decision support. It combines three components: iterative confidence-based modeling, which generates multiple candidate formulations, self-evaluates them across modeling dimensions, and selects one using a max-min confidence criterion; element-level provenance and confidence explanations, which link variables, parameters, constraints, and objectives to quoted source text and provide an audit trail for human verification; and multi-solver routing to specialized optimizer agents for different OR problem classes. Across three OR benchmarks, eight LLM backbones, and four baselines under identical conditions, COOPA achieves the best macro-average accuracy on six of eight backbones and improves over the strongest baseline by up to 6.7 percentage points. A within-system ablation isolates the contribution of iterative confidence-based modeling, while additional analyses and case studies illustrate the value of source traceability and multi-solver dispatch.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

COOPA packages three practical modules into an LLM agent for OR tasks and reports accuracy gains on standard benchmarks, but the gains rest on whether baselines were run under truly matched conditions.

read the letter

The core contribution is a three-part modular agent: iterative confidence-based formulation selection, element-level source provenance with explanations, and routing across specialized solvers. These pieces are combined and tested on three OR benchmarks with eight LLM backbones against four baselines, producing the highest macro-average accuracy on six of the eight models and lifts up to 6.7 points. An ablation isolates the confidence step as useful.

The multi-backbone evaluation is a clear strength; it shows the architecture is not an artifact of one model family. The provenance component directly tackles the opacity problem that limits LLM use in decision support. The abstract states the comparisons were run under identical conditions, which is the right framing.

The main uncertainty is whether those conditions truly equalized the baselines. If the baselines lacked equivalent multi-solver routing or provenance hooks by design, part of the gap could trace to implementation differences rather than the new modules. The three benchmarks are standard but narrow; they do not yet demonstrate coverage of the full range of OR problem classes. The paper would be stronger with explicit confirmation that baseline prompts and interfaces received the same engineering effort as the proposed system.

This is aimed at groups working on LLM agents for structured optimization and planning. Readers who need concrete agent patterns and cross-model numbers will get value from the architecture description and results. The work shows clear thinking about the OR pipeline and honest empirical reporting, so it deserves peer review even with the experimental-control questions that will likely come up.

Referee Report

2 major / 2 minor

Summary. The paper proposes COOPA, a modular LLM-agent architecture for Operations Research problems. It integrates three components: iterative confidence-based modeling (generating and self-evaluating candidate formulations via max-min confidence), element-level provenance and confidence explanations (linking model elements to source text), and multi-solver routing to specialized optimizers. Across three OR benchmarks, eight LLM backbones, and four baselines run under identical conditions, COOPA achieves the best macro-average accuracy on six of eight backbones and improves over the strongest baseline by up to 6.7 percentage points; a within-system ablation isolates the iterative modeling component.

Significance. If the empirical comparisons hold under truly identical conditions, the work offers a concrete, modular advance in LLM-based OR automation by addressing accuracy, interpretability, and solver breadth simultaneously. The provenance and routing mechanisms are particularly relevant for high-stakes domains where auditability matters.

major comments (2)

[Abstract / Experimental Results] Abstract and Experimental Results section: the central claim that COOPA improves accuracy 'under identical conditions' and that gains are attributable to the three proposed components requires explicit confirmation that the four baselines received equivalent prompt templates, solver interfaces, and evaluation protocols. Without tabulated prompt text or interface code for each baseline, it remains possible that the reported 6.7 pp gap arises from unstated differences in baseline adaptation rather than iterative confidence-based modeling or multi-solver routing.
[Ablation study] Ablation study (mentioned in abstract): the within-system ablation isolates iterative confidence-based modeling, but the manuscript must report the exact performance drop when this component is removed for each backbone and benchmark, including statistical significance and variance across runs, to substantiate that the ablation directly supports the attribution of gains.

minor comments (2)

[Experimental setup] The three benchmarks should be named with citations and brief descriptions of their problem classes and sizes in the experimental setup to allow readers to assess representativeness.
[Iterative confidence-based modeling] Notation for the max-min confidence criterion should be formalized (e.g., as an equation) rather than described only in prose.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive feedback emphasizing reproducibility and detailed attribution of results. We address both major comments below and will revise the manuscript to strengthen the claims with additional documentation and data.

read point-by-point responses

Referee: [Abstract / Experimental Results] Abstract and Experimental Results section: the central claim that COOPA improves accuracy 'under identical conditions' and that gains are attributable to the three proposed components requires explicit confirmation that the four baselines received equivalent prompt templates, solver interfaces, and evaluation protocols. Without tabulated prompt text or interface code for each baseline, it remains possible that the reported 6.7 pp gap arises from unstated differences in baseline adaptation rather than iterative confidence-based modeling or multi-solver routing.

Authors: We agree that explicit documentation is required to substantiate identical conditions. The experiments were designed and executed with matched prompt structures, solver APIs, and evaluation metrics across all systems, as stated in the Experimental Setup. To eliminate any ambiguity, the revised manuscript will include a new appendix tabulating the prompt templates for each baseline and COOPA, along with interface pseudocode and evaluation protocol details. This will confirm that observed gains derive from the proposed components rather than implementation differences. revision: yes
Referee: [Ablation study] Ablation study (mentioned in abstract): the within-system ablation isolates iterative confidence-based modeling, but the manuscript must report the exact performance drop when this component is removed for each backbone and benchmark, including statistical significance and variance across runs, to substantiate that the ablation directly supports the attribution of gains.

Authors: We concur that per-backbone and per-benchmark ablation metrics with variance and significance testing are needed for full substantiation. The current within-system ablation demonstrates the contribution of iterative modeling, but the revised manuscript will expand this section with a detailed table reporting accuracy drops for all eight backbones across the three benchmarks when the component is ablated. We will add standard deviations from repeated runs and appropriate statistical tests (e.g., paired t-tests) to quantify significance. revision: yes

Circularity Check

0 steps flagged

No circularity in empirical claims or architecture description

full rationale

The paper's load-bearing claims consist of empirical accuracy measurements on three fixed OR benchmarks against four external baselines, plus a within-system ablation. No equations, first-principles derivations, or predictions appear in the provided text; the architecture components (iterative confidence-based modeling, provenance explanations, multi-solver routing) are described as design choices whose value is assessed by direct measurement rather than by reducing to fitted parameters or self-citations. The abstract explicitly frames results as comparisons 'under identical conditions' on external benchmarks, satisfying the criterion for self-contained evaluation against outside data. No self-citation chains, ansatzes smuggled via citation, or renamings of known results are present. This is the normal non-circular outcome for an empirical systems paper.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 3 invented entities

The paper introduces a new agent architecture built on standard LLM prompting and existing OR solvers; no free parameters, mathematical axioms, or invented physical entities are required beyond the design choices themselves.

invented entities (3)

iterative confidence-based modeling no independent evidence
purpose: Generate and select problem formulations via self-scoring
Core component of the proposed architecture
element-level provenance and confidence explanations no independent evidence
purpose: Link model elements to source text for auditability
Core component of the proposed architecture
multi-solver routing no independent evidence
purpose: Dispatch to specialized optimizer agents by problem class
Core component of the proposed architecture

pith-pipeline@v0.9.1-grok · 5760 in / 1209 out tokens · 48147 ms · 2026-06-29T01:09:54.489585+00:00 · methodology

discussion (0)

Reference graph

Works this paper leans on

23 extracted references

[1]

**Raw Question**: {raw_question}
[2]

**Proposed Formulation**: {formulation_str} Please evaluate the confidence (0-100) for each of the following components:
[3]

**PARAMETERS**: Are all necessary parameters identified with correct values and units?
[4]

**DECISION VARIABLES**: Are all decision variables properly defined with correct domains?
[5]

**OBJECTIVE**: Is the objective function correct and does it properly represent what should be optimized?
[6]

A.3 Refinement Prompt After each iteration, the next candidate is refined using all previous iterations as context

**CONSTRAINTS**: Are all necessary constraints included and correctly formulated? For each component, provide a confidence score from 0-100 and a brief explanation (1-3 sentences). A.3 Refinement Prompt After each iteration, the next candidate is refined using all previous iterations as context. In the default setting, the system completes allk iterations...
[7]

## CRITICAL INSTRUCTIONS: - Your role is solver dispatch

{name} ({sense}): Expression: {expression} Variables: {variables} ... ## CRITICAL INSTRUCTIONS: - Your role is solver dispatch. You MUST NOT solve this problem yourself. - Your ONLY job is to delegate the COMPLETE problem above to the appropriate optimizer agent in your FIRST Code block. - The optimizer agent will handle everything: saving parameters to J...
[8]

Immediately delegate the problem to an optimizer agent in your FIRST Code block
[9]

No internal reasoning, solver code, or calculations

Do not try to solve the problem directly. No internal reasoning, solver code, or calculations
[10]

Do not iterate - Let optimizer agents handle all solution refinement
[11]

=== PROCEDURE ===

Identify agent type and pass complete problem statement to chosen agent. === PROCEDURE ===
[12]

- combinatorial_optimizer_agent for routing, scheduling, CP-SAT, and other discrete problems best expressed in OR-Tools

Select and Delegate to the Appropriate Optimizer: - mathematical_optimizer_agent for algebraic LP, MILP, and continuous NLP models. - combinatorial_optimizer_agent for routing, scheduling, CP-SAT, and other discrete problems best expressed in OR-Tools. - metaheuristic_optimizer_agent for metaheuristic or black-box search, especially multi-objective or non...
[13]

Review and Present Results
[14]

A.6 Mathematical Optimizer Agent Solves algebraic LP, MILP, and continuous NLP problems using Pyomo with GLPK and IPOPT

Call final_answer with the final result. A.6 Mathematical Optimizer Agent Solves algebraic LP, MILP, and continuous NLP problems using Pyomo with GLPK and IPOPT. Mathematical Optimizer System Prompt (core instructions) You are an expert operations research assistant who models and solves mathematical optimization problems using Pyomo in Python, supported ...
[17]

Build the Solver with Pyomo and Save to Python File
[18]

Load and Execute the Solver via load_object_from_python_file()
[20]

A.7 Combinatorial Optimizer Agent Solves VRP, scheduling, assignment, constraint satisfaction, and bin packing problems using Google OR-Tools

Final Answer (only after successful execution). A.7 Combinatorial Optimizer Agent Solves VRP, scheduling, assignment, constraint satisfaction, and bin packing problems using Google OR-Tools. Combinatorial Optimizer System Prompt (core instructions) You are an expert combinatorial optimization assistant specializing in solving combinatorial, routing, and c...
[21]

Understand the Problem
[22]

Create Parameters JSON File
[23]

Build the Solver with OR-Tools, Save to Python File
[24]

Load and Execute via load_object_from_python_file()
[25]

If execution FAILED: fix and re-execute
[26]

The formulation maximizes the sum of efficiencies, but assembly-line capacity is driven by the bottleneck (minimum station rate), so the objective is mis-specified

Final Answer (only after successful execution). A.8 Metaheuristic Optimizer Agent Solves multi-objective, non-convex, and black-box optimization problems using pymoo. 22 Metaheuristic Optimizer System Prompt (core instructions) You are an expert meta-heuristic optimization assistant specializing in solving complex, non-convex, and multi-objective optimiza...

2013

[1] [1]

**Raw Question**: {raw_question}

[2] [2]

**Proposed Formulation**: {formulation_str} Please evaluate the confidence (0-100) for each of the following components:

[3] [3]

**PARAMETERS**: Are all necessary parameters identified with correct values and units?

[4] [4]

**DECISION VARIABLES**: Are all decision variables properly defined with correct domains?

[5] [5]

**OBJECTIVE**: Is the objective function correct and does it properly represent what should be optimized?

[6] [6]

A.3 Refinement Prompt After each iteration, the next candidate is refined using all previous iterations as context

**CONSTRAINTS**: Are all necessary constraints included and correctly formulated? For each component, provide a confidence score from 0-100 and a brief explanation (1-3 sentences). A.3 Refinement Prompt After each iteration, the next candidate is refined using all previous iterations as context. In the default setting, the system completes allk iterations...

[7] [7]

## CRITICAL INSTRUCTIONS: - Your role is solver dispatch

{name} ({sense}): Expression: {expression} Variables: {variables} ... ## CRITICAL INSTRUCTIONS: - Your role is solver dispatch. You MUST NOT solve this problem yourself. - Your ONLY job is to delegate the COMPLETE problem above to the appropriate optimizer agent in your FIRST Code block. - The optimizer agent will handle everything: saving parameters to J...

[8] [8]

Immediately delegate the problem to an optimizer agent in your FIRST Code block

[9] [9]

No internal reasoning, solver code, or calculations

Do not try to solve the problem directly. No internal reasoning, solver code, or calculations

[10] [10]

Do not iterate - Let optimizer agents handle all solution refinement

[11] [11]

=== PROCEDURE ===

Identify agent type and pass complete problem statement to chosen agent. === PROCEDURE ===

[12] [12]

- combinatorial_optimizer_agent for routing, scheduling, CP-SAT, and other discrete problems best expressed in OR-Tools

Select and Delegate to the Appropriate Optimizer: - mathematical_optimizer_agent for algebraic LP, MILP, and continuous NLP models. - combinatorial_optimizer_agent for routing, scheduling, CP-SAT, and other discrete problems best expressed in OR-Tools. - metaheuristic_optimizer_agent for metaheuristic or black-box search, especially multi-objective or non...

[13] [13]

Review and Present Results

[14] [14]

A.6 Mathematical Optimizer Agent Solves algebraic LP, MILP, and continuous NLP problems using Pyomo with GLPK and IPOPT

Call final_answer with the final result. A.6 Mathematical Optimizer Agent Solves algebraic LP, MILP, and continuous NLP problems using Pyomo with GLPK and IPOPT. Mathematical Optimizer System Prompt (core instructions) You are an expert operations research assistant who models and solves mathematical optimization problems using Pyomo in Python, supported ...

[15] [17]

Build the Solver with Pyomo and Save to Python File

[16] [18]

Load and Execute the Solver via load_object_from_python_file()

[17] [20]

A.7 Combinatorial Optimizer Agent Solves VRP, scheduling, assignment, constraint satisfaction, and bin packing problems using Google OR-Tools

Final Answer (only after successful execution). A.7 Combinatorial Optimizer Agent Solves VRP, scheduling, assignment, constraint satisfaction, and bin packing problems using Google OR-Tools. Combinatorial Optimizer System Prompt (core instructions) You are an expert combinatorial optimization assistant specializing in solving combinatorial, routing, and c...

[18] [21]

Understand the Problem

[19] [22]

Create Parameters JSON File

[20] [23]

Build the Solver with OR-Tools, Save to Python File

[21] [24]

Load and Execute via load_object_from_python_file()

[22] [25]

If execution FAILED: fix and re-execute

[23] [26]

The formulation maximizes the sum of efficiencies, but assembly-line capacity is driven by the bottleneck (minimum station rate), so the objective is mis-specified

Final Answer (only after successful execution). A.8 Metaheuristic Optimizer Agent Solves multi-objective, non-convex, and black-box optimization problems using pymoo. 22 Metaheuristic Optimizer System Prompt (core instructions) You are an expert meta-heuristic optimization assistant specializing in solving complex, non-convex, and multi-objective optimiza...

2013