Pith. sign in

REVIEW 3 major objections 6 minor 65 references

LLMs can translate a neutral graph of a fluid system into syntactically valid simulation code, but the generated models are not physically faithful: no Modelica configuration ran directly and the best WNTR run matched the reference about 38

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 07:54 UTC pith:7JX5BEDA

load-bearing objection Solid negative result (LLM simulation fidelity near zero) buried under an overstated syntactic-quality claim because the metrics include manual repairs. the 3 major comments →

arxiv 2607.29389 v1 pith:7JX5BEDA submitted 2026-07-31 cs.LG cs.SE

Simulation Code Generation for Fluid Systems using Large Language Models: Benchmarking Models and Prompting Strategies

Jan Marius St\"urmer , Jascha Knack , Tobias Koch , Andreas Weinmann This is my paper
classification cs.LG cs.SE
keywords LLM code generationsimulation code generationgraph-to-codeprompt engineeringsimulation fidelityWNTRModelicabenchmark
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Seeking to automate the hand-built bridges between engineering graphs and simulation software, the paper asks whether large language models can translate a neutral JSON graph of a fluid system into working wrapper code for two very different environments: the Python library WNTR and the equation-based Modelica language. Across ten LLMs and six prompting strategies, the best configurations do reach acceptable syntactic quality—high pass@1 rates and strong call- and argument-level F1 scores—and the authors find a clear pattern: curated, minimal API snippets beat full codebases or documentation. But the central finding is a negative one: syntactic success does not carry over to simulation. No Modelica configuration produced a directly simulatable model, and the best WNTR configuration reproduced reference trajectories only about 38% of the time. The paper's practical message is that LLM-generated simulation code can accelerate engineering workflows only if treated as drafts requiring manual validation and repair.

Core claim

On the paper's own terms, the discovery is that current LLMs can reliably generate syntactically valid, API-conforming simulation interface layers—the wrapper functions that map each component of a neutral graph to a WNTR or Modelica object—but cannot reliably generate code whose simulated behavior matches a reference. Call- and argument-level F1 scores reach the 0.8–0.9 range for the best models on WNTR, yet average simulation fidelity in the best WNTR configuration is 0.375 ± 0.375, and it is exactly zero for every Modelica configuration. Agentic refinement with a run-only validation function raises pass rates and F1 scores but leaves fidelity at zero for Modelica and essentially unchanged

What carries the argument

The central object is the 'simulation interface layer': a set of wrapper functions, one per component, that translate a labeled directed multigraph (serialized as JSON with explicit types and SI units) into either a WNTR Python network object or a Modelica file. The argument is carried by an evaluation pipeline that checks generated code at four levels: pass@1/pass@3 (does it import and run), call-level and argument-level F1 (do the right API calls and arguments appear), semantic code-embedding similarity, and simulation fidelity (normalized mean absolute error of pressure/flow trajectories against a manually built ground-truth model). The key contrast driving the paper's conclusions is betw

Load-bearing premise

The load-bearing premise is that the manual repairs and the hand-added main() entry point described in Section 3.6 were immaterial to the results; if those edits were frequent or non-trivial, the reported pass@1, F1, and fidelity figures overstate what the LLMs generate on their own.

What would settle it

Take the exact same benchmark and generation prompts, run them with no manual correction and no hand-added entry point, and count how many wrappers import, execute, and reproduce reference trajectories. If pass@1 or simulation fidelity drops materially compared with the paper's numbers, the central claim about current LLM capability is weakened. Even simpler: log and publish every manual edit made per run; a high edit rate on the 'small errors' would settle the question.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Engineers should not treat LLM-generated simulation code as physically trustworthy; the paper's evidence implies every generated model must be re-simulated and compared against a reference before use.
  • Prompt designers should favor small, curated code snippets over full libraries: on both benchmarks the curated prompt outperforms full-code and documentation contexts, consistent with the paper's 'relevance outweighs volume' result.
  • Because domain familiarity appears to dominate model size, a code-specialized model can beat a much larger general model on the language it was trained on, so model choice should depend on the target framework.
  • Agentic refinement with a validation loop improves syntax and API adherence but does not repair physical semantics; adding such agents to a pipeline does not remove the need for human semantic review.
  • For equation-based, strongly typed languages such as Modelica, fully autonomous generation is not yet viable: no configuration in the study produced a directly simulatable model.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The reported numbers are best read as upper bounds on autonomous capability: since all pass@1 and fidelity scores include manual correction of syntax errors and a hand-added main() entry point, a fully hands-off pipeline would almost certainly score lower. The paper gives no count or severity distribution of those repairs, so the size of that gap is unknown.
  • A testable extension follows directly: if the agent's validation function checked physical plausibility (e.g., mass balance or trajectory bounds) instead of only whether the code runs, the agent might iterate toward the kinds of parameter and component-selection errors that currently keep fidelity at zero.
  • The 'relevance over volume' finding suggests that retrieval-augmented prompt construction for simulation code should target canonical calling patterns rather than repository-wide context; injecting too much library source may hurt as much as help.
  • The domain-familiarity hypothesis predicts that a Modelica-trained or Modelica-fine-tuned model should close the WNTR–Modelica gap; that is a concrete benchmark a follow-up could run with the same evaluation suite.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper benchmarks ten LLMs and six prompting strategies for generating 'simulation interface layers' that translate a neutral JSON graph of a fluid system into WNTR (Python) or Modelica code. On a compact benchmark (3 WNTR networks, 4 Modelica modules, 3 runs per configuration), it evaluates pass@1/pass@3, call- and argument-level F1, CodeBERT similarity, and simulation fidelity via NMAE. Results indicate that Kimi-K2.5 and GLM-5.1 are strongest, that curated context outperforms full code/documentation, that Modelica is substantially harder than WNTR, and that simulation fidelity is near zero for Modelica and low for WNTR, even with an OpenCode agent that uses validation feedback. The abstract concludes that the best configurations achieve acceptable syntactic quality but substantial gaps remain in simulation fidelity.

Significance. If fully supported, the study would be a useful reference for practitioners: it provides a broad model/prompt comparison, a multi-metric evaluation, and evidence that agentic validation does not close the physical-fidelity gap. Strengths include the systematic 10-model × 6-strategy design, the combination of static and dynamic validation, and the honest reporting that Modelica fidelity is zero across all configurations. However, because the headline syntactic metrics are computed on human-repaired code with a manually added main() harness, the central positive claim is not yet established. The fidelity-gap conclusion is more robust, since it persists even after human repair.

major comments (3)
  1. [§3.6 and §3.4] All pass@1, F1, and fidelity metrics are computed after the authors manually correct 'small errors like missing parenthesis' and add the entry point def main(network_dict: dict) -> Union[...] (§3.6). The prompts ask for wrapper functions, not an executable program; without this human-added harness, the generated module cannot be run and pass@1 cannot be defined. The paper reports no count, severity distribution, or per-configuration breakdown of manual repairs. The abstract and §4 claim that 'best-performing configurations achieve acceptable syntactic quality' therefore measures a human-assisted pipeline, not raw LLM capability. Please report raw vs. repaired pass@1/F1, quantify the repairs (type, frequency, per model/prompt), and either rephrase the claim or provide evidence that the repairs are immaterial.
  2. [§3.4.1, Eqs. (1)-(2)] The definitions of pass@1 and pass@3 are internally inconsistent. The text says pass@1 'equals the average over the three runs,' but Eq. (1) is 1/N Σ_i E_i with E_i defined as 1 if wrapper i executes successfully with a dummy input; Eq. (2) uses max(E_i,1, E_i,2, E_i,3). This conflates per-wrapper success with per-run success and makes pass@1 indistinguishable from a different quantity (e.g., the fraction of wrappers with at least one success would be pass@3). Clarify the sample unit (wrapper-run vs. wrapper) and report pass@1 as the empirical proportion of individual runs that execute, or give the exact aggregation used.
  3. [§3.3 and §4.4] The 'relevance outweighs volume' conclusion is based on prompt configuration 4, 'Curated/selected code,' which consists of snippets manually extracted by the authors as 'the smallest amount of code that still conveys the logic required for correct wrapper generation' (§3.3). This is not a property of the LLM or a generic prompting strategy; it injects expert knowledge of the exact API calls needed. As a result, the comparison in §4.4 does not isolate relevance as an input variable, and the practical advice to 'carefully narrow down the context' is an instruction to perform manual prompt engineering, not a demonstrated model capability. Please either reframe the claim as 'manually curated snippets outperform full-context prompts' and discuss the human effort involved, or add an automatic retrieval baseline (e.g., RAG) that makes relevance a model-produced variable.
minor comments (6)
  1. [Figure 6] Caption contains typo 'Fideltity' for 'Fidelity'; same spacing issue appears in Table 3/4 header 'A vg.'.
  2. [§3.5] Typo 'EPANAT inp-Files' should be 'EPANET inp files'.
  3. [§4.5] 'We conducted experiences' should read 'We conducted experiments'.
  4. [§3.3] Typo 'wether' should be 'whether'.
  5. [General] No code/data availability statement is provided. Given the manual-repair step, releasing the prompts, JSON schemas, raw generated code, repair logs, and evaluation scripts would be essential for reproducibility and for verifying the magnitude of the human contribution.
  6. [§5] The Discussion already acknowledges that the parser misses class-based wrappers, lowering F1. This is an additional noise source that should be quantified or addressed by extending the parser; otherwise the reported F1 values may underestimate performance unevenly across models.

Circularity Check

0 steps flagged

No circular derivation; the benchmark is an external evaluation. The main validity concern is unquantified manual post-editing in §3.6, which weakens attribution of pass@1/F1 to raw LLM output but is not a circular reduction.

full rationale

The paper's core chain is an empirical measurement, not a derivation from fitted parameters or self-referential definitions. Generated wrappers are compared against independently hand-written ground-truth wrappers using pass@1, F1, CodeBERT similarity, and simulation fidelity (Sections 3.4-3.5). None of these metrics is defined in terms of the LLM outputs it is supposed to validate; the ground-truth wrappers are external reference implementations. The central fidelity-gap claim is also not forced: it persists even in the best configurations (e.g., Modelica fidelity 0.0 in Table 3, WNTR 0.375 +/- 0.375 in Table 4). The two self-citations to Stürmer et al. [8,12] support the upstream P&ID-to-graph digitization step, which is not part of the central benchmarking claim, so they are not load-bearing for the LLM results. The notable weakness is Section 3.6: 'Small errors like missing parenthesis are corrected manually. Further, a single entry point def main(...) ... is added manually.' Since all reported pass@1, F1, and fidelity metrics are computed on these human-repaired, human-extended artifacts, the 'acceptable syntactic quality' claim measures a human-assisted pipeline rather than raw LLM capability. The paper does not report the frequency or severity of these manual corrections, so the assumption that they are immaterial is unverified. However, this is an external-validity and attribution problem, not a circularity: the metric is not constructed to equal an input parameter or a prior conclusion. The OpenCode 'validation function only checks if the code runs' aligns with the pass@1 definition, but the paper transparently describes this loop and does not use it to hide a prediction; its own fidelity results show the limitation. Overall, no specific circular reduction can be exhibited, so the score is low and the flagged concerns belong to correctness/robustness rather than circularity.

Axiom & Free-Parameter Ledger

2 free parameters · 4 axioms · 0 invented entities

The central claim rests on a handful of methodological choices rather than fitted physical parameters: the NMAE tolerance (0.05) and the three-run sampling policy are hand-chosen and directly shape the reported scores; the custom JSON schema and the author-written ground-truth wrappers define what 'correct' means; and the manual post-editing step is an ad hoc procedure whose influence is never measured. No new physical entities are introduced.

free parameters (2)
  • NMAE tolerance ε = 0.05 (5 %)
    Threshold for counting a component's simulated time series as correct (§3.4.3); chosen by hand; directly determines all reported simulation-fidelity values.
  • Runs per configuration k = 3
    pass@k is limited to three runs per configuration due to computational constraints (§3.4.1); with high run-to-run variance (STD up to 0.326 for DeepSeek-V3.2 pass@1 on Modelica), three runs give unstable point estimates.
axioms (4)
  • domain assumption The custom JSON schema (§3.2) is an adequate neutral representation capturing all parameters required by both WNTR and MSL for faithful simulation.
    The schema is built from the intersection of the two frameworks' requirements; if it omits parameters needed for physically accurate simulation, the observed near-zero fidelity is an artifact of specification rather than LLM failure.
  • domain assumption Ground-truth wrappers (§3.5) are minimal, correct, and representative reference implementations.
    All F1 and fidelity scores compare generated code against these author-written wrappers; 'The wrappers were deliberately kept minimal' — non-idiomatic or arbitrary ground truth would penalize valid alternative implementations.
  • ad hoc to paper Manual correction of small syntax errors and manual insertion of main() do not materially change measured capabilities.
    §3.6 introduces human post-editing into the pipeline without quantifying it; the reported pass@1/F1/fidelity are computed on repaired code, so the assumption that repair is immaterial is load-bearing.
  • domain assumption Three generation runs per configuration adequately sample LLM stochasticity.
    §3.6/§4.2: only three runs per configuration; the paper itself reports high standard deviations (e.g., 0.326), undercutting this assumption for weaker models.

pith-pipeline@v1.3.0-daily-deepseek · 18928 in / 11534 out tokens · 125139 ms · 2026-08-03T07:54:22.640394+00:00 · methodology

0 comments
read the original abstract

Large language models (LLMs) have demonstrated a strong ability to generate syntactically correct code from natural-language specifications. In this study, we explore how LLMs can be harnessed to automatically translate a neutral graph representation of fluid system models into executable code for two widely adopted simulation environments: the Python library WNTR and the Modelica Standard Library. We conduct a systematic comparison of ten state-of-the-art LLMs and six prompting strategies that differ in the contextual information supplied (e.g., code or documentation). For each configuration we assess the generated code using a suite of software-quality metrics and we validate the functional fidelity of the resulting simulation models by reproducing benchmark fluid system scenarios. Our findings offer concrete guidance for researchers and engineers seeking to integrate LLM-driven code synthesis into model-based design pipelines. While the best-performing configurations achieve acceptable syntactic quality, we observe substantial gaps remain in simulation fidelity.

Figures

Figures reproduced from arXiv: 2607.29389 by Andreas Weinmann, Jan Marius St\"urmer, Jascha Knack, Tobias Koch.

Figure 1
Figure 1. Figure 1: Pipeline for automatic generation of simulation-interface layers. A neutral graph of a hydraulic system is combined [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: (a) Example WNTR water network topology illustrating basic graph terminology. (b) JSON schema snippet. (c) [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: The instruction asks the model to produce generic [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Overview of the evaluation pipeline. The generated wrapper is first parsed (syntactic check), then token-wise compared [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Token-level comparison between ground-truth (left) and LLM-generated (right) functions. Correct calls and arguments [PITH_FULL_IMAGE:figures/full_fig_p009_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Pearson correlation matrix (absolute values) of [PITH_FULL_IMAGE:figures/full_fig_p011_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Performance of LLMs on WNTR wrapper generation. Each model is run for prompt configurations 1 (zero-shot) to 5 [PITH_FULL_IMAGE:figures/full_fig_p012_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Performance of LLMs on Modelica wrapper generation. Each model is run for prompt configurations 1 (zero-shot) to [PITH_FULL_IMAGE:figures/full_fig_p013_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: WNTR simulation fidelity results: (left) average across three runs, (right) best of three runs. The consistently low [PITH_FULL_IMAGE:figures/full_fig_p014_9.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

65 extracted references · 14 canonical work pages · 2 internal anchors

  1. [1]

    Banks, J

    J. Banks, J. Carson, B. Nelson, D. Nicol, Discrete- event system simulation, 5. ed. ed., Pearson, Pren- tice Hall, Upper Saddle River, NJ u.a., 2010. URL: https://permalink.obvsg.at/AC08313798

    2010

  2. [2]

    Novák, E

    P. Novák, E. Serral, R. Mordinyi, R. Šindelář, In- tegrating heterogeneous engineering knowledge and tools for efficient industrial simulation model sup- port, Advanced Engineering Informatics 29 (2015) 575–

    2015

  3. [3]

    Makahleh, A

    F. Makahleh, A. Nassar, Modeling and simulation of fluid flow and energy systems, in: A. Abou Jaoudé (Ed.), Simulation modeling, IntechOpen, London, 2024. doi:10.5772/intechopen.1002220

    work page doi:10.5772/intechopen.1002220 2024

  4. [4]

    Lampropoulos, X

    G. Lampropoulos, X. Larrucea, R. Colomo-Palacios, Digital twins in critical infrastructure, Information 15 (2024) 454. URL: https://www.mdpi.com/2078-2489/ 15/8/454. doi:10.3390/info15080454

    work page doi:10.3390/info15080454 2024

  5. [5]

    B. J. Sattler, J. Friesen, A. Tundis, P. F. Pelz, Modeling and validation of residential water demand in agent- based models: A systematic literature review, Water 15 (2023) 579. doi:10.3390/w15030579

    work page doi:10.3390/w15030579 2023

  6. [6]

    F. E. Cellier, E. Kofman, Continuous System Simulation, SpringerLink Bücher, Springer Science+Business Media Inc, Boston, MA, 2006. doi:10.1007/0-387-30260-3. 16

    work page doi:10.1007/0-387-30260-3 2006

  7. [7]

    W. L. Oberkampf, S. M. DeLand, B. M. Ruther- ford, K. V. Diegert, K. F. Alvin, Error and uncertainty in modeling and simulation, Reli- ability Engineering & System Safety 75 (2002) 333–357. URL: https://www.sciencedirect. com/science/article/pii/S095183200100120X. doi:10.1016/S0951-8320(01)00120-X

    work page doi:10.1016/s0951-8320(01)00120-x 2002

  8. [8]

    J. M. Stürmer, M. Graumann, T. Koch, From engineer- ing diagrams to graphs: Digitizing p&ids with transform- ers, in: 2025 IEEE 12th International Conference on Data Science and Advanced Analytics (DSAA), IEEE, Piscataway, NJ, 2025, pp. 1–11. doi:10.1109/DSAA65442. 2025.11248012

    arXiv 2025

  9. [9]

    Pohlmann, M

    U. Pohlmann, M. Tichy, Modelica code generation from modelicaml state machines extended by asynchronous communication, Proceedings of the 4th International Workshop on Equation-Based Object-Oriented Modeling Languages and Tools (2011). URL:https://ep.liu.se/ konferensartikel.aspx?issue=56&Article_No=9

    2011

  10. [10]

    Nytsch-Geusen, A

    C. Nytsch-Geusen, A. Inderfurth, W. Kaul, K. Mucha, J. Rädler, M. Thorade, C. Ribas Tugores, Template based code generation of modelica building energy simu- lation models, in: Proceedings of the 12th International Modelica Conference, Prague, Czech Republic, May 15- 17, 2017, Linköping Electronic Conference Proceedings, Linköping University Electronic Pr...

    2017

  11. [11]

    G. S. Martínez, S. A. Sierla, T. A. Karhela, J. Lap- palainen, Viatkin Valeriy, Automatic generation of a high-fidelity dynamic thermal-hydraulic process simula- tion model from a 3d plant model, IEEE Access 6 (2018) 45217–45232. doi:10.1109/ACCESS.2018.2865206

    arXiv 2018

  12. [12]

    J. M. Stürmer, M. Graumann, T. Koch, Demon- strating automated generation of simulation models from engineering diagrams, in: 2023 International Conference on Machine Learning and Applications (ICMLA), IEEE, 12/15/2023 - 12/17/2023, pp. 1156–

    2023

  13. [13]

    Xiong, H

    X. Xiong, H. Cai, H. Yu, B. Shen, P. Hu, Dr- rag: Domain-rule-based retrieval-augmented generation for aviation digital model design, Advanced Engineering Informatics 68 (2025) 103688. URL: https://www.sciencedirect. com/science/article/pii/S1474034625005816. doi:10.1016/j.aei.2025.103688

    arXiv 2025

  14. [14]

    M. Chen, J. Tworek, H. Jun, Q. Yuan, Pinto, Henrique Ponde de Oliveira, J. Kaplan, H. Edwards, Y. Burda, N. Joseph, G. Brockman, A. Ray, R. Puri, G. Krueger, M. Petrov, H. Khlaaf, G. Sastry, P. Mishkin, B. Chan, S. Gray, N. Ryder, M. Pavlov, A. Power, L. Kaiser, M. Bavarian, C. Winter, P. Tillet, F. P. Such, D. Cum- mings, M. Plappert, F. Chantzis, E. Bar...

    Pith/arXiv arXiv doi:10.48550/arxiv.2107.03374 2021

  15. [15]

    H. Jin, H. Chen, Are llms reliable code review- ers? systematic overcorrection in requirement con- formance judgement, Automated Software Engineer- ing 33 (2026) 90. URL:https://link.springer.com/ article/10.1007/s10515-026-00638-5. doi: 10.1007/ s10515-026-00638-5

    work page doi:10.1007/s10515-026-00638-5 2026

  16. [16]

    ModiGen: A Large Language Model-Based Workflow for Multi-Task Modelica Code Generation

    J. Xiang, T. Ye, P. Liu, Y. Zhang, W. Wang, Modigen: A large language model-based workflow for multi-task mod- elica code generation, arXiv.org (2025). URL:https:// arxiv.org/pdf/2503.18460. doi:10.48550/arXiv.2503. 18460

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2503.18460 2025

  17. [17]

    X. Ren, Q. Zang, Z. Guo, Simugen: Multi-modal agentic framework for constructing block diagram-based simu- lation models, arXiv.org (2025). URL:https://arxiv. org/pdf/2506.15695. doi:10.48550/arXiv.2506.15695

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2506.15695 2025

  18. [18]

    Klise, D

    K. Klise, D. Hart, M. Bynum, J. Hogge, T. Hax- ton, R. Murray, J. Burkhardt, Water network tool for resilience (wntr) user manual: Version 0.2.3, 2020. URL: https://cfpub.epa.gov/si/si_public_record_ Report.cfm?dirEntryId=349798&Lab=CESER

    2020

  19. [19]

    URL: https://modelica.org/

    Modelica Association, Modelica, 24.11.2025. URL: https://modelica.org/

    2025

  20. [20]

    Jiang, F

    J. Jiang, F. Wang, J. Shen, S. Kim, S. Kim, A survey on large language models for code generation, ACM Transactions on Software Engineering and Methodology (2025). doi:10.1145/3747588

    doi:10.1145/3747588 2025

  21. [21]

    J. Chen, S. Chen, J. Cao, J. Shen, S.-C. Cheung, When retrieval augmentation meets api documentation: Can llms code with less-common libraries?, ACM Transac- tions on Software Engineering and Methodology (2026). doi:10.1145/3821424

    work page doi:10.1145/3821424 2026

  22. [22]

    Huynh, B

    N. Huynh, B. Lin, Large language models for code generation: A comprehensive survey of chal- lenges, techniques, evaluation, and applications, arXiv.org (2025). URL:https://arxiv.org/pdf/2503. 01245. doi:10.48550/arXiv.2503.01245

    Pith/arXiv arXiv doi:10.48550/arxiv.2503.01245 2025

  23. [23]

    Bistarelli, M

    S. Bistarelli, M. Fiore, I. Mercanti, M. Mongiello, Usage of large language model for code genera- tion tasks: A review, SN Computer Science 6 (2025) 673. URL: https://link.springer.com/ article/10.1007/s42979-025-04241-5. doi: 10.1007/ s42979-025-04241-5

    doi:10.1007/s42979-025-04241-5 2025

  24. [24]

    Zhang, B

    F. Zhang, B. Chen, Y. Zhang, J. Keung, J. Liu, D. Zan, Y. Mao, J.-G. Lou, W. Chen, Repocoder: Repository- level code completion through iterative retrieval and generation, arXiv.org(2023).URL: https://arxiv.org/ pdf/2303.12570. doi:10.48550/arXiv.2303.12570

    Pith/arXiv arXiv doi:10.48550/arxiv.2303.12570 2023

  25. [25]

    Y. Li, E. Shi, D. Zheng, K. Duan, J. Chen, Y. Wang, Re- pomincoder: Improving repository-level code generation based on information loss screening, in: H. Mei, J. Lv, A. Helal, X. Ma, S.-C. Cheung, J. Zhang, T. Zhang (Eds.), Proceedings of the 15th Asia-Pacific Symposium on Internetware, ACM Digital Library, Association for Computing Machinery, Erscheinun...

    arXiv 2024

  26. [26]

    Y. Gao, Y. Xiong, X. Gao, K. Jia, J. Pan, Y. Bi, Y. Dai, J. Sun, M. Wang, H. Wang, Retrieval- augmented generation for large language models: A survey, arXiv.org (2023). URL: https://arxiv.org/ pdf/2312.10997. doi:10.48550/arXiv.2312.10997

    Pith/arXiv arXiv doi:10.48550/arxiv.2312.10997 2023

  27. [27]

    H. Han, Y. Wang, H. Shomer, K. Guo, J. Ding, Y. Lei, M. Halappanavar, R. A. Rossi, S. Mukherjee, X. Tang, Q. He, Z. Hua, B. Long, T. Zhao, N. Shah, A. Javari, Y. Xia, J. Tang, Retrieval-augmented generation with graphs (graphrag), arXiv.org (2024). URL: https:// arxiv.org/pdf/2501.00309. doi:10.48550/arXiv.2501. 00309

    Pith/arXiv arXiv doi:10.48550/arxiv.2501.00309 2024

  28. [28]

    J. Li, X. Shi, K. Zhang, G. Li, Z. Jin, L. Li, H. Zhang, F. Liu, Y. Zhang, Z. Tao, Y. Dong, Y. Zhu, C. Tao, Graphcodeagent: Dual graph-guided llm agent for retrieval-augmented repo-level code genera- tion, arXiv.org (2025). URL:https://arxiv.org/pdf/ 2504.10046. doi:10.48550/arXiv.2504.10046

    doi:10.48550/arxiv.2504.10046 2025

  29. [29]

    Zhang, J

    K. Zhang, J. Li, G. Li, X. Shi, Z. Jin, Codeagent: Enhancing code generation with tool-integrated agent systems for real-world repo-level coding challenges, Pro- ceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (2024) 13643–13658. URL:https://aclanthology.org/ 2024.acl-long.737/. doi:10.18653/v1/...

    doi:10.18653/v1/2024.acl-long 2024

  30. [30]

    URL: https://opencode.ai/

    anomalyco, Opencode ai coding agent, 2026. URL: https://opencode.ai/

    2026

  31. [31]

    Drath (Ed.), AutomationML: Das Lehrbuch Für Studium und Praxis, De Gruyter Studium Series, 1st ed

    R. Drath (Ed.), AutomationML: Das Lehrbuch Für Studium und Praxis, De Gruyter Studium Series, 1st ed. ed., Walter de Gruyter GmbH, Basel/Berlin/Boston, 2022

    2022

  32. [32]

    Tolksdorf, D

    G. Tolksdorf, D. B. Cameron, M. Theißen, Dexpi 2.0: Synergistic integration of pfd and p&id in a unified digital model, Chemie Ingenieur Technik 97 (2025) 1065–

    2025

  33. [33]

    L. A. Rossman, An Overview of EPANET Version 3.0, American Society of Civil Engineers, 2012. doi:10.1061/ 41203(425)3

    2012

  34. [34]

    Kaiser, A

    B. Kaiser, A. Reichle, A. Verl, Model-based automatic generation of digital twin models for the simulation of reconfigurable manufacturing sys- tems for timber construction, Procedia CIRP 107 (2022) 387–392. URL: https://www.sciencedirect. com/science/article/pii/S2212827122002797. doi:10. 1016/j.procir.2022.04.063

    2022

  35. [35]

    Bjørnskov, M

    J. Bjørnskov, M. Jradi, M. Wetter, Auto- mated model generation and parameter estima- tion of building energy models using an ontology- based framework, Energy and Buildings 329 (2025) 115228. URL: https://www.sciencedirect. com/science/article/pii/S0378778824013446. doi:10. 1016/j.enbuild.2024.115228

    arXiv 2025

  36. [36]

    M. Mans, T. Blacha, T. Schreiber, D. Müller, De- velopment and application of an open-source frame- work for automated thermal network generation and simulations in modelica, Energies 15 (2022) 4372. doi:10.3390/en15124372

    work page doi:10.3390/en15124372 2022

  37. [37]

    Bahamdan, E

    A. Bahamdan, E. Pajak, J. D. Hedengren, A. R. Del Chanona, Sketch2simulation: Automating flow- sheet generation via multi agent large language mod- els (2026). URL: https://arxiv.org/pdf/2603.24629. doi:10.48550/arXiv.2603.24629

    doi:10.48550/arxiv.2603.24629 2026

  38. [38]

    Y. Xia, D. Dittler, N. Jazdi, H. Chen, M. Weyrich, Llm experiments with simulation: Large language model multi-agent system for simulation model parametriza- tion in digital twins, in: 2024 IEEE 29th Interna- tional Conference on Emerging Technologies and Fac- tory Automation, IEEE, Piscataway, NJ, 2024, pp. 1–4. doi:10.1109/ETFA61755.2024.10710900

    arXiv 2024

  39. [39]

    Rupprecht, Y

    S. Rupprecht, Y. Hounat, M. Kumar, G. Lastrucci, A. M. Schweidtmann, Text2model: Generating dynamic chemical reactor models using large language models (llms), in: J. van Impe, G. Lonard, S. S. Bhonsale, M. Polanska, F. Logist (Eds.), Proceedings of the 35th European Symposium on Computer Aided Process En- gineering (ESCAPE 35), Systems and Control Transa...

    work page doi:10.69997/sct.165009 2025

  40. [40]

    Z. Dong, Z. Lu, Y. Yang, Fine-tuning a large language model for automating computational fluid dynamics simulations, Theoretical and Applied Mechanics Letters 15 (2025) 100594. URL: https://www.sciencedirect. com/science/article/pii/S2095034925000261. doi:10. 1016/j.taml.2025.100594

    arXiv 2025

  41. [41]

    Svajlenko, C

    J. Svajlenko, C. K. Roy, A survey on the evalua- tion of clone detection performance and benchmarking, arXiv.org (2020). URL:https://arxiv.org/pdf/2006. 15682. doi:10.48550/arXiv.2006.15682

    Pith/arXiv arXiv doi:10.48550/arxiv.2006.15682 2020

  42. [42]

    Martinez-Gil, Source code clone detection using unsupervised similarity measures, in: P

    J. Martinez-Gil, Source code clone detection using unsupervised similarity measures, in: P. Bludau, R. Ramler, D. Winkler, J. Bergsmann (Eds.), Soft- ware Quality as a Foundation for Security, Lecture Notes in Business Information Processing, Springer Nature Switzerland and Imprint Springer, Cham, 2024, pp. 21–37. URL: https://link.springer.com/ chapter/1...

    work page doi:10.1007/978-3-031-56281-5_2 2024

  43. [43]

    Möltner, P

    T. Möltner, P. Manzl, M. Pieber, J. Gerstmayr, Cre- ation, evaluation and self-validation of simulation mod- els with large language models, Neurocomputing 663 (2026) 132030. URL:https://www.sciencedirect. com/science/article/pii/S092523122502702X. doi:10. 1016/j.neucom.2025.132030

    arXiv 2026

  44. [44]

    Z. Feng, D. Guo, D. Tang, N. Duan, X. Feng, M. Gong, L. Shou, B. Qin, T. Liu, D. Jiang, M. Zhou, Codebert: A pre-trained model for programming and natural languages, Findings of the Association for Computational Linguistics: EMNLP 2020 (2020) 1536–1547. URL: https:// aclanthology.org/2020.findings-emnlp.139/. doi:10. 18653/v1/2020.findings-emnlp.139

    2020

  45. [45]

    Moddemann, J

    L. Moddemann, J. Ehrhardt, A. Diedrich, O. Nigge- mann, The hai-cpps benchmark: Evaluating ai capa- bilities across hybrid data spaces, 2025 IEEE 30th International Conference on Emerging Technologies and Factory Automation (ETFA) (2025) 1–8. doi:10.1109/ ETFA65518.2025.11205680. 18

    arXiv 2025

  46. [46]

    R. Cao, M. Chen, J. Chen, Z. Cui, Y. Feng, B. Hui, Y. Jing, K. Li, M. Li, J. Lin, Z. Ma, K. Shum, X. Wang, J. Wei, J. Yang, J. Zhang, L. Zhang, Z. Zhang, W. Zhao, F. Zhou, Qwen3-coder-next technical report, 2026. URL: https://arxiv.org/pdf/2603.00729

    Pith/arXiv arXiv 2026

  47. [47]

    Agarwal, L

    OpenAI, S. Agarwal, L. Ahmad, J. Ai, et. al, gpt-oss- 120b & gpt-oss-20b model card, 2025. URL:https:// arxiv.org/pdf/2508.10925

    Pith/arXiv arXiv 2025

  48. [48]

    Rastogi, A

    A. Rastogi, A. Yang, A. Q. Jiang, et. al, Devstral: Fine- tuning language models for coding agent applications,

  49. [49]

    URL: https://www.minimax

    MiniMax, Minimax m2.5: Built for real-world produc- tivity, MiniMax (2026). URL: https://www.minimax. io/news/minimax-m25

    2026

  50. [50]

    URL:https://arxiv

    DeepSeekAI, Deepseek-v3.2: Pushing the frontier of open large language models, 2025. URL:https://arxiv. org/pdf/2512.02556

    Pith/arXiv arXiv 2025

  51. [51]

    GLM-5 Team:, A. Zeng, X. Lv, Z. e. Hou, Glm-5: from vibe coding to agentic engineering, 2026. URL:https: //arxiv.org/pdf/2602.15763

    Pith/arXiv arXiv 2026

  52. [52]

    Kimi Team, T. Bai, Y. Bai, Y. Bao, et. al, Kimi k2.5: Visual agentic intelligence, 2026. URL:https://arxiv. org/pdf/2602.02276

    Pith/arXiv arXiv 2026

  53. [53]

    URL:https://openai.com/ de-DE/index/introducing-gpt-5/

    OpenAI, Gpt-5, 24.11.2025. URL:https://openai.com/ de-DE/index/introducing-gpt-5/

    2025

  54. [54]

    Pichai, D

    S. Pichai, D. Hasabis, K. Kavukcuoglu, A new era of intelligence with gemini 3, Google (2025). URL: https://blog.google/products-and-platforms/ products/gemini/gemini-3/

    2025

  55. [55]

    URL: https://www-cdn.anthropic.com/ bbd8ef16d70b7a1665f14f306ee88b53f686aa75/Claude% 20Sonnet%204.6%20System%20Card.pdf

    Anthropic, Claude sonnet 4.6 system card (2026). URL: https://www-cdn.anthropic.com/ bbd8ef16d70b7a1665f14f306ee88b53f686aa75/Claude% 20Sonnet%204.6%20System%20Card.pdf

    2026

  56. [56]

    Ashkboos, I

    S. Ashkboos, I. Markov, E. Frantar, T. Zhong, X. Wang, J. Ren, T. Hoefler, D. Alistarh, Quik: Towards end-to- end 4-bit inference on generative large language mod- els, Proceedings of the 2024 Conference on Empiri- cal Methods in Natural Language Processing (2024) 3355–3371. URL: https://aclanthology.org/2024. emnlp-main.197/. doi: 10.18653/v1/2024.emnlp-...

    doi:10.18653/v1/2024.emnlp-main 2024

  57. [57]

    URL:https://github.com/ggml-org/llama.cpp

    ggml org, llama.cpp: Llm inference in c/c++, 2026. URL:https://github.com/ggml-org/llama.cpp

    2026

  58. [58]

    R. B. Pereira, A. Plastino, B. Zadrozny, L. H. Merschmann, Correlation analysis of performance measures for multi-label classification, Informa- tion Processing & Management 54 (2018) 359–

    2018

  59. [59]

    N. F. Liu, K. Lin, J. Hewitt, A. Paranjape, M. Bevilacqua, F. Petroni, P. Liang, Lost in the middle: How language models use long contexts, Transactions of the Association for Computa- tional Linguistics 12 (2024) 157–173. URL: https: //aclanthology.org/2024.tacl-1.9/. doi: 10.1162/ tacl{\textunderscore}a{\textunderscore}00638. 19

    2024

  60. [207]

    doi:10.3384/ecp17132199

    work page doi:10.3384/ecp17132199

  61. [369]

    doi: 10.1016/j.ipm

    URL: https://www.sciencedirect.com/science/ article/pii/S0306457318300165. doi: 10.1016/j.ipm. 2018.01.002

    doi:10.1016/j.ipm 2018

  62. [590]

    doi: 10.1016/j.aei

    URL: https://www.sciencedirect.com/science/ article/pii/S1474034615000555. doi: 10.1016/j.aei. 2015.05.001

    doi:10.1016/j.aei 2015

  63. [1069]

    doi:10.1002/cite.70009

    work page doi:10.1002/cite.70009

  64. [1162]

    doi:10.1109/ICMLA58977.2023.00173

    arXiv 2023

  65. [2025]

    URL:https://arxiv.org/pdf/2509.25193

    arXiv