Pith. sign in

REVIEW 4 major objections 5 minor 20 references

Fine-tuning small language models on grammar-corrupted and cross-model error-correction data teaches them to generate executable MiniZinc code, lifting execution accuracy from near zero to 98% in an ensemble, while solution accuracy remains

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Fine-tuning small LMs on synthetic and bootstrapped syntax-error corrections lifts MiniZinc execution accuracy from ~0% to 98% in an ensemble, but solution accuracy saturates near 35%.

T0 review reviewed 2026-08-02 challenge →

load-bearing objection Useful, credible fine-tuning study for MiniZinc with a clever error-correction data recipe; the main weakness is that the IndustryOr test-set exclusion is asserted but not auditable. the 4 major comments →

arxiv 2607.20456 v1 pith:TSCRWRFL submitted 2026-05-14 cs.CL cs.AI

Learn2Zinc: Fine-tuning Small Language Models for Text-to-Model Translation in MiniZinc

Serdar Kadioglu , Karthik Uppuluri This is my paper
classification cs.CL cs.AI
keywords MiniZincsmall language modelsfine-tuningcode generationerror correctionconstraint programmingtext-to-model translationsyntax errors
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether small language models (0.6B–20B parameters) can be taught to generate code in MiniZinc, a rare domain-specific language for constraint modeling. The authors find that out-of-the-box execution accuracy for these models is 0–6%, with syntax errors dominating, and they argue this is because MiniZinc is nearly absent from pretraining corpora. They build a fine-tuning dataset that combines verified text–model pairs, grammar-based corruptions of MiniZinc code, and cross-model error bootstrapping—real errors produced by one model and corrected by another. After fine-tuning, the best single model reaches 76% execution accuracy, self-reflection pushes it to 89%, and an ensemble of five fine-tuned small models reaches 98% execution accuracy. Solution accuracy, however, saturates near 35%, so the paper's conclusion is that syntax is learnable through targeted fine-tuning while constraint reasoning remains the bottleneck.

Core claim

The central discovery is that the syntax gap for a rare domain-specific language like MiniZinc can be closed by fine-tuning small language models on an error-correction objective, rather than adding more code examples or chain-of-thought reasoning. The paper demonstrates this by showing that an augmented fine-tuning dataset—8,014 base generation pairs, plus 4,452 synthetic corruptions and 2,286 cross-model bootstrapped corrections—converts near-zero execution accuracy into 98% for an ensemble of five fine-tuned models (Qwen3-0.6B, LLaMA-3.2-1B/3B, Gemma-2-9B, GPT-OSS-20B). The same treatment leaves solution accuracy at 34–35%, which the authors interpret as separating the syntactic problem f

What carries the argument

The central mechanism is 'cross-model error bootstrapping': run several fine-tuned small language models at multiple sampling temperatures, collect execution failures, and use a frontier model to produce minimal fixes that pass compilation; these (corrupted code, corrected code) pairs become training examples, teaching the small model to avoid and repair its own syntax errors. This is complemented by a grammar-based corruption taxonomy derived from the MiniZinc BNF (20 rules in three difficulty levels). The error-correction supervision is what differentiates the Augmented strategy from plain generation fine-tuning, and it is what enables self-reflection loops to fix mistakes at inference tim

Load-bearing premise

The load-bearing premise is that the 100-problem Text2Zinc-IndustryOr test set is truly disjoint from all fine-tuning data, including the 15,649 augmented pairs and the bootstrapping runs; the paper asserts this but gives no overlap detection or deduplication evidence, so a hidden near-duplicate would inflate the reported 98% execution accuracy.

What would settle it

Compute near-duplicate similarity between each of the 100 IndustryOr test problems and the 8,014 base pairs plus 15,649 augmented pairs (character n-grams or semantic embeddings); if any test problem or close variant is present in the training data, the 98% execution result is inflated. Alternatively, re-run the ensemble on a freshly curated held-out MiniZinc set from the same distribution and check whether execution accuracy drops materially from 98%.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • If the 98% execution result is real, the syntactic barrier for using small models as MiniZinc assistants is effectively removed; the remaining obstacle is semantic correctness.
  • Fine-tuned small models, when ensembled, can exceed the execution accuracy of a frontier model with agentic prompting on this benchmark, suggesting that domain-specific fine-tuning is a viable alternative to large-scale prompting for rare languages.
  • The finding that chain-of-thought fine-tuning underperforms base fine-tuning implies that reasoning traces cannot substitute for missing syntax knowledge at small scale; syntax should be taught before reasoning.
  • Once code compiles, retries do not fix semantic errors, so solution accuracy is governed by modeling decisions, not compilation success—this separates the two problems cleanly.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A natural extension is to run the same error-bootstrapping recipe on other model-oriented DSLs (e.g., GAMS, CPMpy): if the syntax gap closes there too while semantic accuracy stays low, the syntax/semantics split is general, not MiniZinc-specific.
  • The two dominant semantic failure modes the paper describes—contradictory constraints and 'phantom' variables that force zero objectives—suggest a concrete next experiment: add training examples that introduce and repair these semantic flaws, and measure whether solution accuracy moves.
  • Because the reported 98% rests on the test set's disjunction from training data, an independent contamination audit (e.g., n-gram overlap between test and all augmented pairs) is the cheapest check that the result is not an artifact.
  • The cross-scale bootstrapping pattern—smaller models produce the most corrections and learn from larger models' errors—hints at a distillation loop: repeatedly bootstrap errors from the fine-tuned ensemble to create ever-stronger training data.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes Learn2Zinc, a fine-tuning pipeline for small language models (0.6B–20B parameters) to generate MiniZinc constraint models from natural-language problem descriptions. The authors construct training data from Text2Zinc verified solutions and Or-Instruct problems translated to MiniZinc via GPT-5.2, then augment it with grammar-based synthetic corruptions and cross-model error bootstrapping. They evaluate five SLMs on a 100-problem Text2Zinc-IndustryOr test set and report that augmented fine-tuning improves execution accuracy over direct generation and chain-of-thought variants, that self-reflection further improves execution accuracy, and that a top-down ensemble reaches 98% execution accuracy while solution accuracy remains near 35%.

Significance. If the results are robust, the paper makes a useful contribution: it documents a concrete syntax bottleneck for a rare DSL, shows that targeted fine-tuning can mitigate that bottleneck even with sub-10B models, and releases data, models, and code. The verification protocol—compiling MiniZinc models and checking objective values against ground truth—is a genuine strength, as is the open release of the pipeline. The main scientific value lies in the separation of syntactic learnability from semantic reasoning, which is clearly demonstrated internally. However, the headline 98% execution claim rests entirely on the test set having been excluded from every stage of the fine-tuning data construction, and the manuscript currently provides only an assertion of that exclusion. The statistical presentation also lacks any variance estimates, and the ensemble method is not specified to the level needed for reproducibility.

major comments (4)
  1. [§4.4 / §2.2 / Table 1] The load-bearing claim that the 100 Text2Zinc-IndustryOr test problems are not in the fine-tuning data is only asserted, not demonstrated. §2.2 states that Text2Zinc includes problems from IndustryOr, Mamo, and Nl4Opt; §4.4 asserts 'theses problems are not included in the fine-tuning'; Table 1 lists training sources with no IndustryOr row. No split hashes, problem-ID overlap lists, or near-duplicate analyses are provided. Because the 8,014 base pairs, the 4,452 synthetic corruptions, and the 2,286 cross-model corrections are all built from the same Text2Zinc pool, any test problem or near-duplicate in training would directly inflate the 98% execution accuracy. Please provide release-quality split metadata, exact problem IDs, and a deduplication check against all 15,649 fine-tuning examples.
  2. [§6.2 / Table 8] The 98% ensemble result is not reproducible from the written description. The text says models are 'tried in descending order of capability' with retries, but it does not specify how many retries each model receives, whether self-reflection is used at each cascade level, whether a problem counts as successful when any model produces an executable model, how ties are handled, or how the 98% execution accuracy is aggregated across the 100 problems. Please provide algorithm-level pseudocode, per-problem results, and the exact criterion used to compute ensemble execution accuracy.
  3. [§4.4 / Tables 2–8] All results are single-run percentages on 100 test problems, with no confidence intervals, seeds, or repeated fine-tuning/decoding runs. With n=100, the standard error of a proportion near 50% is about 5 percentage points; differences such as 70% vs. 74% or 49% vs. 51% are not meaningful without variance estimates. This matters particularly for the claims that CoT underperforms Base across all models and that augmented fine-tuning consistently outperforms Base. Please report binomial confidence intervals and ideally repeated runs with different seeds.
  4. [§5.1 / §3] The augmented data is generated with GPT-5.2 both for Copt-to-MiniZinc translation and for correcting bootstrapped errors, creating a distributional coupling between the training data and the frontier-model outputs used for comparison. This is not circular, but it limits the claim that the pipeline 'teaches syntax' from independent signal. An ablation isolating synthetic corruptions only vs. GPT-5.2-corrected examples would clarify how much of the execution gain is attributable to the proposed cross-model bootstrapping as opposed to generic correction examples.
minor comments (5)
  1. [§1] The introduction says 'four out of five models... Qwen3, LLaMa, and Gemini' — only three model names are listed, and Gemini is not part of the evaluated model set. The tested models are Qwen3, LLaMa, Gemma, and GPT-OSS.
  2. [§5.1 / Table 6] The text above Table 6 says 'cross-model error correction bootstrapping instances (6,728)' but Table 6 reports 6,738 and 6,738 + 8,911 = 15,649. The 6,728 figure is a typo.
  3. [§5.2 / Table 2] The text says Qwen3-0.6B jumps to 65% execution accuracy with augmented fine-tuning, but Table 2 reports 64.0%. The narrative and table should be aligned.
  4. [§10 / Abstract] The conclusion says solution accuracy saturates at 34%, while the abstract and Table 8 say 35%. Use one consistent figure.
  5. [§4.4] Typo: 'theses problems' should be 'these problems'. Also, 'opens-source' in the abstract should be 'open-source'.

Circularity Check

1 steps flagged

No equation-level circularity in the fine-tuning derivation; the only substantial concern is train/test provenance, which the manuscript asserts but does not fully verify.

specific steps
  1. other [§2.2 (Text2Zinc description), §3 (dataset construction), §4.4 (test set), Table 1]
    "For the remaining problems, which originate from IndustryOr, Mamo, and Nl4Opt, ground-truth objective values are available... Our fine-tuning dataset starts from the verified MiniZinc solutions in Text2Zinc... For benchmarking, we use Text2Zinc-IndustryOr (100 problems) as our test set. It is important to note theses problems are not included in the fine-tuning."

    Section 2.2 states Text2Zinc contains problems originating from IndustryOr; Section 3 constructs the 8,014 base pairs from verified Text2Zinc MiniZinc solutions; Section 4.4 then tests on Text2Zinc-IndustryOr and asserts exclusion. Table 1, however, lists no IndustryOr row and no overlap/deduplication evidence is provided. If IndustryOr is part of Text2Zinc as §2.2 says, the test set is a subset of the training pool by construction, and the 98% execution accuracy would be a memorization artifact rather than a syntax-learning prediction. The manuscript's only counter-evidence is the asserted sentence, so the central quantitative claim is not independently verifiable from the paper. This is a data-provenance gap rather than a demonstrated equation-level reduction.

full rationale

The derivation chain is otherwise self-contained and not circular: the fine-tuning data are built from verified MiniZinc solutions (Text2Zinc) plus Or-Instruct translations, then augmented with grammar-based corruptions and cross-model bootstrapping. Execution accuracy is checked by an external compiler/solver, not by a fitted constant relabeled as a prediction. Self-reflection reuses the error-correction task format from training, but that is normal supervised generalization on a held-out benchmark, not a by-construction reduction. Self-citations to Text2Zinc and Text2Model are data/baseline references, not the result itself; no uniqueness theorem or ansatz is imported from prior work. The only load-bearing caveat is the IndustryOr exclusion: the paper's own Section 2.2 suggests Text2Zinc contains IndustryOr problems, Section 3 derives training pairs from Text2Zinc, Table 1 omits IndustryOr, and Section 4.4 merely asserts exclusion without split hashes or overlap checks. If a leak occurred, the 98% claim would collapse to memorization, but this remains an unverified data-hygiene risk rather than a demonstrated circularity, so the overall circularity score is low.

Axiom & Free-Parameter Ledger

2 free parameters · 4 axioms · 0 invented entities

The paper introduces no new theoretical entities; its central empirical claim rests on data-quality and split assumptions. The most important assumption, auditable train/test separation, is asserted but not demonstrated.

free parameters (2)
  • Corruption-rule difficulty mix = 30% easy / 30% medium / 30% hard / 10% identity
    Hand-chosen in §5.1; no ablation or sensitivity analysis is reported, yet it directly shapes the augmented fine-tuning dataset and therefore the reported execution gains.
  • Self-reflection and ensemble budget = up to 5 attempts per model; 5-model top-down ensemble
    Self-reflection limit and ensemble order are arbitrary choices; the total LLM-call cost of the 98% ensemble is not reported, making budget comparisons with GPT-5.2 baselines unclear.
axioms (4)
  • domain assumption Text2Zinc ground-truth objectives and manually written MiniZinc models are correct
    Used as the verification signal to filter training pairs and to decide whether a generated MiniZinc model is acceptable (§2.2, §3).
  • domain assumption GPT-5.2 repairs are minimal and preserve problem semantics
    Cross-model bootstrapping relies on GPT-5.2 to fix each corrupted model with 'only the minimal changes necessary'; no human audit of the repaired models is reported (§5.1).
  • domain assumption IndustryOr test set is disjoint from all fine-tuning data
    Explicitly asserted in §4.4 but not demonstrated with split identifiers; Table 1 does not list IndustryOr among training sources, and no overlap detection is provided.
  • domain assumption BNF corruption rules approximate real LLM syntax errors
    Synthetic corruption uses 20 grammar-derived rules; cross-model bootstrapping attempts to compensate for gaps, but the paper does not measure how well the synthetic errors match actual test-time errors (§5.1).

reviewed 2026-08-02 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Learn2Zinc: Fine-tuning Small Language Models for Text-to-Model Translation in MiniZinc." pith.science (2026). https://pith.science/paper/TSCRWRFL

@misc{pith2026260720456,
  author       = {Pith},
  title        = {Pith review of: Learn2Zinc: Fine-tuning Small Language Models for Text-to-Model Translation in MiniZinc},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TSCRWRFL}},
  note         = {Machine review of arXiv:2607.20456}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Large language models excel at code generation for mainstream programming languages but struggle with rare, domain-specific languages such as MiniZinc, a constraint modeling language for combinatorial problems. We investigate whether targeted fine-tuning can teach small language models (0.6B to 20B parameters) to generate syntactically correct and semantically valid MiniZinc models from natural language problem descriptions. Our key finding is that syntax errors dominate failures when working with this domain specific language: the out-of-the-box execution accuracy of small language models such as Qwen3, LLaMa, Gemma, and GPT-OSS is near-zero. We propose a cross-model error bootstrapping approach that collects syntax errors from multiple LLM runs and leverage those to curate an error correction training dataset. This dataset allows us fine-tune small language models that consistently improves both direct code generation and chain-of-thought approaches across all model sizes. With self-reflection and ensembling, our approach achieves up to 98\% execution accuracy. In parallel, solution accuracy still remains at 35\%, indicating that while syntax is learnable, constraint reasoning remains a challenge. We contribute our fine-tuning pipeline, datasets, and models to opens-source for further research on text-to-model translation.

Figures

Figures reproduced from arXiv: 2607.20456 by Karthik Uppuluri, Serdar Kadioglu.

Figure 1
Figure 1. Figure 1: Execution accuracy (%) across fine-tuning strategies and model sizes. [PITH_FULL_IMAGE:figures/full_fig_p006_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Top-15 MiniZinc grammar corruption rules color coded by difficulty. 5.1 Error Correction Dataset For error correction fine-tuning, we draw data samples from two different strategies; synthetic corruption and cross-model error bootstrapping. Synthetic Corruptions. We leverage the Backus-Naur Form (BNF) grammar of MiniZinc6 to derive 20 corruption rules, each grounded in a specific grammar production. These … view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

20 extracted references · 3 canonical work pages

  1. [5]

    Tias Guns

    URLhttps://arxiv.org/abs/2502.13776. Tias Guns. Increasing modeling language convenience with a universal n-dimensional array, cppy as python- embedded example. InProceedings of the 18th workshop on Constraint Modelling and Reformulation at CP (Modref 2019), volume 19,

    arXiv 2019

  2. [6]

    URLhttps://arxiv.org/abs/2405.17743. Md. Shahidul Islam, Fariha Mamud, Rakib Ul Haque, and Ahmed Y. Saber. Automatic formulation and optimization of linear problems from a structured paragraph. InProc. 2021 Int. Conf. on Science & Contemporary Technologies (ICSCT),

    Pith/arXiv arXiv 2021

  3. [10]

    Kuo Liang, Yuhang Lu, Jianming Mao, Shuyi Sun, Chunwei Yang, Congcong Zeng, Xiao Jin, Hanzhang Qin, Ruihao Zhu, and Chung-Piaw Teo

    URLhttps://arxiv.org/abs/2311.15271. Kuo Liang, Yuhang Lu, Jianming Mao, Shuyi Sun, Chunwei Yang, Congcong Zeng, Xiao Jin, Hanzhang Qin, Ruihao Zhu, and Chung-Piaw Teo. Large-scale optimization model auto-formulation: Harnessing llm flexibility via structured workflow,

    Pith/arXiv arXiv

  4. [11]

    KostisMichailidis,DimosTsouros,andTiasGuns.ConstraintModellingwithLLMsUsingIn-ContextLearning

    URLhttps://arxiv.org/abs/2601.09635. KostisMichailidis,DimosTsouros,andTiasGuns.ConstraintModellingwithLLMsUsingIn-ContextLearning. In30th International Conference on Principles and Practice of Constraint Programming (CP 2024), volume 307 ofLeibniz International Proceedings in Informatics (LIPIcs), pp. 20:1–20:27. Schloss Dagstuhl – Leibniz-Zentrum für In...

    arXiv 2024

  5. [12]

    Nicholas Nethercote, Peter J

    doi: 10.4230/LIPIcs.CP.2024.20. Nicholas Nethercote, Peter J. Stuckey, Ralph Becket, Sebastian Brand, Gregory J. Duck, and Guido Tack. Minizinc: Towards a standard cp modelling language. In Christian Bessière (ed.),Principles and PracticeofConstraintProgramming–CP2007,pp.529–543,Berlin,Heidelberg,2007.SpringerBerlin Heidelberg. ISBN 978-3-540-74970-7. Ope...

    doi:10.4230/lipics.cp.2024.20 2024

  6. [13]

    RindraRamamonjison,TimothyYu,LinziXing,MahdiMostajabdaveh,XiaoruiLi,XiaojinFu,XiongweiHan, Yuanzhe Chen, Ren Li, Kun Mao, and Yong Zhang

    URLhttps://arxiv.org/abs/2303.08774. RindraRamamonjison,TimothyYu,LinziXing,MahdiMostajabdaveh,XiaoruiLi,XiaojinFu,XiongweiHan, Yuanzhe Chen, Ren Li, Kun Mao, and Yong Zhang. LaTeX2Solver: a hierarchical semantic parsing of LaTeXdocumentintocodeforanassistiveoptimizationmodelingapplication. InProceedingsofthe61st Annual Meeting of the Association for Comp...

    Pith/arXiv arXiv 2023

  7. [15]

    David Simchi-Levi, Tinglong Dai, Ishai Menache, and Michelle Xiao Wu

    URLhttps://arxiv.org/abs/2603.09678. David Simchi-Levi, Tinglong Dai, Ishai Menache, and Michelle Xiao Wu. Democratizing optimization with generative ai.Johns Hopkins Carey Business School Research Paper Forthcoming, October 24

    Pith/arXiv arXiv

  8. [16]

    Text2zinc: A cross-domain dataset for modeling optimizationandsatisfactionproblemsinminizinc,2025

    Akash Singirikonda, Serdar Kadıoğlu, and Karthik Uppuluri. Text2zinc: A cross-domain dataset for modeling optimizationandsatisfactionproblemsinminizinc,2025. URLhttps://arxiv.org/abs/2503. 10642. 15 Gemini Team et al. Gemini: A family of highly capable multimodal models,

    2025

  9. [17]

    org/abs/2312.11805

    URLhttps://arxiv. org/abs/2312.11805. Dimos Tsouros, Helene Verhaeghe, Serdar Kadıoğlu, and Tias Guns. Holy grail 2.0: From natural language to constraint models.arXiv preprint arXiv:2308.01589,

    Pith/arXiv arXiv

  10. [18]

    Ziyang Xiao, Dongxiang Zhang, Yangjun Wu, Lilin Xu, Yuan Jessica Wang, Xiongwei Han, Xiaojin Fu, Tao Zhong, Jia Zeng, Mingli Song, et al

    doi: 10.1609/aaai.v39i27.35090. Ziyang Xiao, Dongxiang Zhang, Yangjun Wu, Lilin Xu, Yuan Jessica Wang, Xiongwei Han, Xiaojin Fu, Tao Zhong, Jia Zeng, Mingli Song, et al. Chain-of-experts: When llms meet complex operations research problems. InThe Twelfth International Conference on Learning Representations,

    work page doi:10.1609/aaai.v39i27.35090

  11. [19]

    Optimind: Teachingllmstothinklikeoptimizationexperts,2026

    Xinzhi Zhang, Zeyi Chen, Humishka Zope, Hugo Barbalho, Konstantina Mellou, Marco Molinaro, Janardhan Kulkarni,IshaiMenache,andSiruiLi. Optimind: Teachingllmstothinklikeoptimizationexperts,2026. URLhttps://arxiv.org/abs/2509.22979. Max Zuo, Francisco Piedrahita Velez, Xiaochen Li, Michael L. Littman, and Stephen H. Bach. Planetarium: A rigorous benchmark f...

    arXiv 2026

  12. [20]

    A","B","C

    URLhttps: //arxiv.org/abs/2407.03321. 16 A Fine-tuning Examples Learn2Zinc-Base Example Instruction: GenerateMiniZinccode for the following optimization problem. A rescue material distribution company needs to allocate rescue materials to three disaster- stricken areas: A, B, and C. These places have different degrees of disaster, so the required amount o...

    arXiv

  13. [2004]

    doi: 10.1007/978-1-4613-0215-5_8

    ISBN 978-1-4613-0215-5. doi: 10.1007/978-1-4613-0215-5_8. Junyang Cai, Serdar Kadıoğlu, and Bistra Dilkina. Gala: Global llm agents for text-to-model translation,

    work page doi:10.1007/978-1-4613-0215-5_8

  14. [2020]

    Caigao Jiang, Xiang Shu, Hong Qian, Xingyu Lu, Jun Zhou, Aimin Zhou, and Yang Yu

    doi: 10.18653/v1/2020.findings-emnlp.100. Caigao Jiang, Xiang Shu, Hong Qian, Xingyu Lu, Jun Zhou, Aimin Zhou, and Yang Yu. Llmopt: Learning to define and solve general optimization problems from scratch. InProceedings of the 13th International Conference on Learning Representations (ICLR), Singapore,

    work page doi:10.18653/v1/2020.findings-emnlp.100 2020

  15. [2021]

    ElgunJabrayilzadeandSelmaTekir.LGPSolver–solvinglogicgridpuzzlesautomatically.InFindingsoftheAs- sociation for Computational Linguistics: EMNLP 2020, pp

    doi: 10.1109/ICSCT53883.2021.9642516. ElgunJabrayilzadeandSelmaTekir.LGPSolver–solvinglogicgridpuzzlesautomatically.InFindingsoftheAs- sociation for Computational Linguistics: EMNLP 2020, pp. 1118–1123. Association for Computational Linguistics, November

    arXiv 2021

  16. [2022]

    Aman Sharma and Paras Chopra

    URLhttps://arxiv.org/abs/2209.15565. Aman Sharma and Paras Chopra. Esolang-bench: Evaluating genuine reasoning in large language models via esoteric programming languages,

    Pith/arXiv arXiv

  17. [2023]

    Deepseek-v3technicalreport,2025

    DeepSeek-AIetal. Deepseek-v3technicalreport,2025. URLhttps://arxiv.org/abs/2412.19437. Alex Duchnowski, Ellie Pavlick, and Alexander Koller. Ehop: A dataset of everyday np-hard optimization problems,

    Pith/arXiv arXiv 2025

  18. [2024]

    Michael R

    URLhttps: //arxiv.org/abs/2407.19633. Michael R. Bussieck and Alex Meeraus.General Algebraic Modeling System (GAMS), pp. 137–157. Springer US, Boston, MA,

    Pith/arXiv arXiv

  19. [2025]

    14 ParagPravinDakle,SerdarKadıoğlu,KarthikUppuluri,ReginaPoliti,PreethiRaghavan,SaiKrishnaRallabandi, and Ravisutha Srinivasamurthy

    URLhttps://arxiv.org/abs/2509.08970. 14 ParagPravinDakle,SerdarKadıoğlu,KarthikUppuluri,ReginaPoliti,PreethiRaghavan,SaiKrishnaRallabandi, and Ravisutha Srinivasamurthy. Ner4opt: Named entity recognition for optimization modelling from natural language. InInternational Conference on Integration of Constraint Programming, Artificial Intelligence, and Opera...

    arXiv

  20. [2026]

    MichaelKlamkin,ArnaudDeza,SikaiCheng,HaoruoZhao,andPascalVanHentenryck

    URLhttps://arxiv.org/abs/2604.12955. MichaelKlamkin,ArnaudDeza,SikaiCheng,HaoruoZhao,andPascalVanHentenryck. Dualschool: Howre- liablearellmsforoptimizationeducation?,2025. URLhttps://arxiv.org/abs/2505.21775. Qingyang Li, Lele Zhang, and Vicky Mak-Hau. Synthesizing mixed-integer linear programming models from natural language descriptions,

    Pith/arXiv arXiv 2025

This paper was first reviewed by deepseek-v4-flash on August 2, 2026.