{"id":"911000d1-7280-4d0d-ac63-9a8eeef71755","arxiv_id":"2501.00568","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"ChatGPT, when given tailored prompts containing the rules of robust optimization, can reformulate simple linear programs into robust and adaptive robust forms and generate solving code, but the evidence is limited to a few examples.","lead":"This paper shows that with carefully written prompts, ChatGPT can be guided to add uncertainty to simple linear optimization problems, derive their robust counterparts, and generate Julia code to solve them. The work is a proof-of-concept that large language models might lower the barrier to using robust optimization, though the evidence is anecdotal and reliability is not yet measured.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The paper's central 'little to no errors' claim is unquantified and contradicted by the paper's own Section 3 error examples; without a systematic evaluation the demonstrated successes cannot be treated as representative.","rationale":"The strongest claim is an empirical performance claim, so the load-bearing condition is that the displayed successes are typical rather than selected. The reader's weakest_assumption identifies exactly this. My independent check found the paper supplies no error rate; moreover, Section 3's documented mistakes on simple instances put the 'little to no errors' statement in direct tension. This is an internal-consistency problem, not merely a disagreement with external consensus. The proposed concrete test would settle it directly: it operationalizes 'simple problems' as a finite test distribution and measures pass rates. Since the evidence is a useful proof-of-concept but not yet a measured claim, I agree with the CONDITIONAL verdict. No change to the reader's verdict is required.","tokens_in":23998,"tokens_out":2211,"duration_ms":21551,"concrete_test":"Re-run the exact four-step pipelines (Prompts 2.1-2.6) on a pre-registered test suite of 20 simple LO problems, covering the categories used in §2 (custom constraints, diet, network flow) crossed with the uncertainty sets in §2.1.1 (p-norm, polyhedral, intersection, Minkowski sum), repeating each prompt 5 times to account for stochasticity. Score each output blinded against a known-correct robust counterpart and runnable Julia solution; count per-step pass rates for (i) uncertainty introduction, (ii) robust counterpart derivation, (iii) code executability, and (iv) correct optimal value. If the average per-step pass rate is not near 100% (e.g., below 90%), the 'little to no errors' claim fails and the contribution reduces to existence proofs for a few hand-picked prompts.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The strongest claim asserts that with appropriate prompting ChatGPT performs the RO/ARO pipeline 'with little to no errors' for simple problems. The evidence consists of a handful of curated successful conversations in §2, with no sampling rule, no scoring rubric, and no error count. Section 3 then reports concrete failures on equally simple inputs: it gives the wrong algebraic rearrangement for 100x1+x2≥10+x1 ([−99,1] instead of [−99,−1]), uses Minkowski-sum combination instead of intersection when taking the RC of Z1∩Z2, and shows that identical prompts yield different responses. These are exactly the 'simple' settings the central claim is about, so the assertion 'little to no errors' is not merely under-supported; the paper's own evidence shows non-negligible error propensity. For the central claim to hold, the successful examples would need to be shown representative, e.g., via a defined test distribution and measured pass rates. That is missing, so the empirical core of the paper is currently unverified.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper reports a proof-of-concept study in which ChatGPT 3.5, guided by specialized informational prompts and chain-of-thought prompting, is used to automate parts of the Robust Optimization (RO) and Adaptive Robust Optimization (ARO) modeling pipeline: problem formulation, introduction of uncertainty, derivation of the robust counterpart, and generation of executable Julia code. The authors present four worked example conversations, describe the prompts they designed, and then list limitations including inconsistency, algebraic errors, and incorrect application of theorems. The paper's title and abstract claim that, for simple problems, ChatGPT performs these steps 'with little to no errors,' while later sections acknowledge concrete failures on equally simple inputs.","tokens_in":24278,"tokens_out":4616,"duration_ms":48866,"significance":"If the central claim were rigorously established, the paper would provide a useful demonstration that LLMs can be steered into a niche technical domain through in-context prompting, potentially lowering the barrier to using RO/ARO. The authors are transparent about their failures and about the stochastic nature of the model, and they give a clear description of the prompting recipe, which is a useful starting point for further work. However, the evidence is anecdotal: there is no systematic test set, no error rate, no baseline comparison, and no reproducibility protocol. These omissions are load-bearing because the contribution rests on the reliability of the claimed capability, not on the mathematical correctness of the background material, which is standard. The paper is best read as a qualitative demonstration; in its current form the empirical core is unverified.","major_comments":[{"comment":"The abstract's claim that ChatGPT performs the RO/ARO steps 'with little to no errors' is not supported by any quantitative evidence. No test set, sampling rule, scoring rubric, or error count is given. Section 3 itself reports concrete errors on simple inputs: the constraint 100x1 + x2 >= 10 + x1 is rearranged to a = [-99, 1] instead of the correct [-99, -1], and the robust counterpart for the intersection of a 2-norm ball and an infinity-norm ball is incorrectly derived using the Minkowski-sum property rather than the intersection rule with auxiliary variables w1 and w2. These are precisely the simple settings to which the abstract's claim refers. The authors should either conduct a systematic evaluation with defined pass/fail criteria or substantially qualify the claim.","section":"Abstract; §2.1.2 and §2.2.2"},{"comment":"The paper asserts that ChatGPT performs 'very poorly out-of-the-box' on RO and ARO and that the proposed in-context prompting is what enables the demonstrated behavior, but no baseline is reported. The reader cannot determine whether the successes are due to the prompts, to the model's general capabilities, or to the particular examples chosen. To support the adaptation claim, the authors should run the same tasks without the specialized prompts (e.g., the same user queries with no appended Prompt 2.1-2.6) and report both outcomes.","section":"§1.1, §1.2, §2.1.2"},{"comment":"The demonstration is not reproducible as reported. The examples in §2 appear to be selected successful conversations, but the selection criterion is not stated, and no version/date of ChatGPT, temperature setting, number of repeated runs, or failure counts are provided. Section 3 itself notes that identical prompts yield different responses, so a single transcript is not a stable artifact. For an INFORMS Journal on Computing submission, the authors should specify the model version and sampling parameters, run each example multiple times, and provide transcripts or a public repository with the prompts and generated code.","section":"§2 and §3"}],"minor_comments":[{"comment":"The abstract contains the typo 'higlhy' for 'highly', and 'Massachussetts' appears in the author affiliations; these should be corrected.","section":"Abstract; title page"},{"comment":"In Eq. (11), the dual norm notation ||P x||_E with E = D/(D-1) is introduced without explaining that this is the dual norm of the original D-norm ball; this should be stated explicitly.","section":"§2.1.1, Eq. (11)"},{"comment":"The phrase 'conjugate of the support function' is a misnomer: the quantity δ*(x|Z) is the support function of Z, i.e., the conjugate of the indicator function of Z. This terminology should be corrected, since the prompts are a key part of the method and accuracy of mathematical language matters.","section":"Prompt 2.3 and Prompt 2.6"},{"comment":"The user prompt contains the typo '≤=' instead of '≤'; also, when describing the correct intersection robust counterpart, the auxiliary-variable constraint w1 + w2 = P^T x is only mentioned in prose and not displayed in the final corrected form, which would improve clarity.","section":"§3, 'Applying mathematical theorems' example"}],"recommendation":"major_revision","confidential_remarks":"The stress-test concern lands: the paper's own Section 3 contradicts the abstract's 'little to no errors' claim. Given the journal's emphasis on reproducible computational work, the anecdotal evidence is a serious shortfall. The mathematical background is standard and the prompt designs are interesting, so the manuscript could be made publishable by adding a systematic evaluation and baselines, or by honestly reframing the paper as a qualitative demonstration. I do not see grounds for ethical concerns; the limitations are disclosed candidly."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is the first paper I know of that pushes ChatGPT into robust and adaptive robust optimization specifically, and the authors are honest enough to show failures. The new bit is the domain application: NL4OPT and Tsouros et al. did general optimization or constraint modeling, but nobody had demonstrated the full robust-counterpart pipeline via in-context prompting. The prompts are printed in full, so the method is reproducible, and the examples are genuinely instructive—ChatGPT handling the >= to <= flip, different uncertainty sets per constraint, linear decision rules, and even extrapolating to a quadratic decision rule.\n\nThe soft spot is the central claim. The abstract and intro say ChatGPT performs these steps with \"little to no errors\" for simple problems, but there is no test set, no sampling rule, no error count. Section 3 then shows the model tripping on simple algebra (100x1 + x2 >= 10 + x1 becomes a = [-99, 1] instead of [-99, -1]) and confusing Minkowski sum with intersection when combining uncertainty sets. Those are exactly the simple settings the claim is about. So the successes in Section 2 cannot be read as representative without a defined test distribution and measured pass rates. The stress-test note is right on this.\n\nAlso, the prompts hand the model the transformation rules, so the demonstration shows the model can apply rules it is given, not that it discovered RO. That is fine for a proof-of-concept, but it lowers the wow factor. The paper itself acknowledges this indirectly by saying ChatGPT performs poorly out-of-the-box.\n\nWho is this for: people working on LLMs for operations research, and readers curious whether ChatGPT can be scaffolded into niche domains with heavy prompting. It is a proof-of-concept, not a benchmark. As an IJOC paper it is light; as a workshop paper or a first study it is fine.\n\nRecommendation: send it to peer review, but the referee should require systematic evaluation—a defined problem distribution, pass rates, and an error taxonomy—before the \"little to no errors\" claim can stand. If the authors reframe the contribution as \"ChatGPT can do this with substantial scaffolding, and here are the failure modes,\" it is a decent contribution.","headline":"First LLM-for-RO/ARO proof-of-concept with honest failure examples, but the 'little to no errors' claim is unquantified and the evidence is anecdotal.","tokens_in":24623,"tokens_out":2264,"would_cite":true,"duration_ms":22460,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"With tailored prompts, ChatGPT can reformulate simple robust optimization problems into solvable code.","keywords":["robust optimization","adaptive robust optimization","large language models","ChatGPT","in-context learning","prompt engineering","support functions","optimization formulation"],"falsifier":"Run the paper's prompting pipeline on a fixed benchmark of, say, one hundred simple robust linear programs (a few variables, one or two constraints, standard uncertainty sets such as $\\ell_1$, $\\ell_2$, $\\ell_\\infty$, and polyhedral), execute the generated Julia code, and compare each robust counterpart against a reference derivation; the central claim weakens if more than a small fraction of simple cases produce an incorrect robust counterpart or fail to run.","tokens_in":23788,"feed_emoji":"🤖","tokens_out":5944,"duration_ms":53814,"temperature":0.7,"pith_summary":"Robust and adaptive optimization usually demands a modeling expert who can identify uncertainty sets, apply duality and support-function results, and produce tractable reformulations. This paper claims that ChatGPT, guided by specialized in-context prompts and chain-of-thought instructions, can perform the whole pipeline on simple problems: formulate an optimization problem, add user-specified uncertainty, derive the robust counterpart, and generate executable Julia code. The authors argue the approach works with little to no errors on simple cases, while acknowledging that harder cases and algebraic manipulations can fail. If the claim holds, it would let non-experts apply robust optimization without learning its mathematical toolkit, and would show that large language models can adapt to narrow technical domains through prompt engineering alone.","feed_headline":"ChatGPT learns robust optimization from a few tailored prompts","feed_subtitle":"Prompt context teaches the chatbot to add uncertainty, derive robust counterparts, and write solving code.","key_machinery":"The engine of the approach is a set of prompts that package the support-function calculus of robust optimization as rule tables. For a constraint $(a+Pz)^\\top x \\le b$ with $z$ in an uncertainty set $Z$, the robust counterpart is $a^\\top x + \\sigma_Z^*(P^\\top x) \\le b$, where $\\sigma_Z^*$ is the conjugate support function of $Z$; the prompts supply closed forms for $p$-norm balls, polyhedral sets, intersections, and Minkowski sums, plus a chain-of-thought instruction to apply them step by step. A second prompt set tells the model how to linearize $\\ell_1$ and $\\ell_\\infty$ norms and to model $\\ell_2$ norms with second-order cone constraints when writing solver code. For adaptive robust optimization, the prompts introduce the linear decision rule $y(z)=v+Uz$, which converts an adjustable constraint into a standard robust constraint with the same support-function machinery. The entire method is the combination of these rule tables with in-context learning, not any new optimization theory.","core_discovery":"The paper's central demonstration is that a general-purpose chatbot, given a small set of informational prompts encoding the vocabulary of robust and adaptive optimization, can act as an automatic RO/ARO modeler. In the worked examples, ChatGPT correctly converts a constraint of the form $(a+Pz)^\\top x \\le b$ into its tractable robust counterpart $a^\\top x + \\sigma_Z^*(P^\\top x) \\le b$ for choices such as $p$-norm and polyhedral uncertainty sets, correctly handles constraints given in $\\ge$ form by flipping signs, combines uncertainty sets in some cases, and writes executable Julia code that linearizes $\\ell_1$ and $\\ell_\\infty$ norms and models $\\ell_2$ norms via second-order cones. For adaptive problems, it formulates wait-and-see decisions under a linear decision rule $y(z)=v+Uz$, takes the robust counterpart of the resulting single-stage problem, and even extrapolates to a quadratic decision rule. The paper also documents failures: inconsistent answers across runs, algebraic rearrangement errors, and using the Minkowski-sum rule instead of the intersection rule when combining uncertainty sets. The overall claim is that in-context learning plus chain-of-thought prompting is sufficient adaptation to this niche domain, even though ChatGPT without those prompts performs poorly out of the box.","pith_inferences":["A systematic benchmark would likely show that success rates drop sharply as problem size or uncertainty-set complexity grows; the paper's examples are small and hand-picked.","The hybrid pattern suggested here—LLM generates, a solver or symbolic engine verifies—could make the approach reliable enough for real use; the paper itself stops at standalone prompting.","The same prompt-encoding strategy could be applied to other transformation-heavy fields, such as conic duality, stochastic programming reformulations, or integer-programming cuts, where a compact rule table is the key bottleneck for non-experts."],"forward_implications":["Non-expert users could describe a problem in plain language and receive a robust model with executable code, without learning support functions or duality theory.","The prompting scheme could be used as an interactive teaching aid for robust and adaptive optimization, since ChatGPT walks through each reformulation step.","If the method generalizes, simple RO/ARO projects that currently require a modeling specialist could be automated, lowering the cost of adding robustness.","For adaptive problems, the same prompts let the model handle here-and-now versus wait-and-see variables under linear decision rules, a key class of practical dynamic problems.","Because the model can extrapolate to quadratic decision rules from a linear-rule example, the approach may extend to richer decision-rule families without new prompts."],"supporting_citations":[{"why":"Supplies the support-function calculus and robust counterpart formulas that the prompts encode, including linear decision rules for adaptive optimization.","marker":"Bertsimas and Den Hertog (2022)"},{"why":"Provides chain-of-thought prompting, the reasoning technique the paper adopts to get step-by-step derivations from ChatGPT.","marker":"Wei et al. (2023)"},{"why":"The NL4OPT competition demonstrates ChatGPT's existing ability to formulate optimization problems from natural language, which this paper extends to robust counterparts.","marker":"Ramamonjison et al. (2023)"},{"why":"Documents the algebraic and theorem-application weaknesses of ChatGPT that the paper uses to explain the observed failures.","marker":"Frieder et al. (2023)"},{"why":"A prior position paper proposing ChatGPT for constraint-model generation; the paper's prompt-engineering approach builds on that idea.","marker":"Tsouros et al. (2023)"}],"fun_headline_variants":["ChatGPT auto-formulates and solves robust optimization","Prompted ChatGPT derives robust counterparts and code","LLM few-shot learning for robust optimization","ChatGPT turns uncertainty into optimization code","Robust optimization via ChatGPT: prompts matter"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The demonstrations are a small selection of conversations, and the paper reports no systematic measurements of how often the pipeline succeeds, so the 'little to no errors' claim rests on the authors' informal observation of a few successful runs.","fun_headline_variants_meta":{"raw":{"variants":["ChatGPT auto-formulates and solves robust optimization","Prompted ChatGPT derives robust counterparts and code","LLM few-shot learning for robust optimization","ChatGPT turns uncertainty into optimization code","Robust optimization via ChatGPT: prompts matter"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001248,"raw_usage":{"total_tokens":5131,"prompt_tokens":973,"completion_tokens":4158,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":589,"completion_tokens_details":{"reasoning_tokens":4092}},"tokens_in":589,"tokens_out":4158,"duration_ms":27648,"temperature":1.0,"reasoning_tokens":4092,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T22:46:38.692919+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the paper's prompting pipeline on a fixed benchmark of, say, one hundred simple robust linear programs (a few variables, one or two constraints, standard uncertainty sets such as $\\ell_1$, $\\ell_2$, $\\ell_\\infty$, and polyhedral), execute the generated Julia code, and compare each robust counterpart against a reference derivation; the central claim weakens if more than a small fraction of simple cases produce an incorrect robust counterpart or fail to run.","supporting_citations":[{"cited_title":"https://www.dynamic-ideas.com/books/robust-and-adaptive-optimization","cited_arxiv_id":null,"evidence_quote":"Supplies the support-function calculus and robust counterpart formulas that the prompts encode, including linear decision rules for adaptive optimization."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The NL4OPT competition demonstrates ChatGPT's existing ability to formulate optimization problems from natural language, which this paper extends to robust counterparts."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Documents the algebraic and theorem-application weaknesses of ChatGPT that the paper uses to explain the observed failures."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"A prior position paper proposing ChatGPT for constraint-model generation; the paper's prompt-engineering approach builds on that idea."}],"review_version":1}