Investigating the Role of LLMs Hyperparameter Tuning and Prompt Engineering to Support Domain Modeling

Antinisca Di Marco; Claudio Di Sipio; Davide Di Ruscio; Giordano d'Aloisio; Vladyslav Bulhakov

REVIEW 3 major objections 6 minor 41 references

Investigating the Role of LLMs Hyperparameter Tuning and Prompt Engineering to Support Domain Modeling

T0 review · 3 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper tries to establish that search-based tuning of an LLM's decoding hyperparameters, combined with few-shot or chain-of-thought prompts, makes Llama 3.1 produce domain models closer to the ground truth than default settings on most…

desk verdict A solid, honest empirical skeleton under an overstated conclusion: tuning helps the text-similarity proxy, but the semantic-quality claim doesn't survive Table 4. read the letter →

arxiv 2507.14735 v1 pith:D7QQPL3S submitted 2025-07-19 cs.SE

Vladyslav Bulhakov , Giordano d'Aloisio , Claudio Di Sipio , Antinisca Di Marco , Davide Di Ruscio This is my paper

classification cs.SE

keywords domainmodelinglargelanguagemodelshyperparametertuningpromptengineeringNSGA-IILlama3.1Ecoremetamodelmodel-driven

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to show that the decoding settings of a large language model can be tuned, without fine-tuning, to generate better domain models from text. It runs a two-stage search—NSGA-II followed by grid search—over Llama 3.1 hyperparameters (temperature, top-k, top-p, repetition penalty, max new tokens), using one medical domain model (LIFEMap) as the tuning target and Cosine Similarity plus BERTScore as quality measures. Six optimal configurations emerge, and the paper then tests them across ten application domains under zero-shot, few-shot, and chain-of-thought prompts, comparing against Llama 3.1 with default settings. Its central result is that hyperparameter tuning alone does not generalize across domains, but tuning combined with prompt engineering beats the default baseline on most domains, with few-shot prompting improving both syntactic and semantic similarity and chain-of-thought improving syntactic similarity most. If correct, this gives modelers a cheap alternative to fine-tuning: tune once, reuse the settings with a suitable prompt.

What carries the argument

The load-bearing mechanism is the two-stage search pipeline for LLM decoding hyperparameters, with the Pareto front as its output. NSGA-II first shrinks a wide hyperparameter space to a reduced range; an exhaustive grid search then evaluates every combination in that reduced space to produce six non-dominated configurations, each a trade-off between Cosine Similarity (syntactic correctness) and BERTScore (semantic correctness). These configurations are the objects reused across domains, and the prompt—zero-shot, few-shot, or chain-of-thought—is the second variable that determines whether the tuned settings transfer.

What would settle it

Take the ten domains and the six tuned configurations, generate models with few-shot and chain-of-thought prompts, then parse the outputs as Ecore and compare them to the ground truth with a structural model-difference tool that compares Ecore elements directly. If the configurations that win on Cosine Similarity and BERTScore are not also the ones with the highest structural similarity, or if default hyperparameters match ground-truth structure as often as tuned ones, the paper's central claim that tuning and prompt engineering improve generated domain models is refuted.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that optimal decoding hyperparameters found on a single reference domain transfer to other domains once a prompt strategy is added. The sanity check (RQ1.1) shows tuned configurations significantly outperform default Llama 3.1 on the LIFEMap model in both Cosine Similarity and BERTScore. The generalization test (RQ1.2) shows that with zero-shot prompts no tuned configuration wins more often than it loses, so tuning alone is not reusable. When few-shot or chain-of-thought prompts are added (RQ2), the tuned configurations produce significantly better models than the default baseline on a majority of the ten domains; few-shot is the only strategy that raises both syntactic and semantic similarity, while chain-of-thought gives the best syntactic similarity. The paper therefore claims that hyperparameter tuning plus prompt engineering is an effective, low-cost substitute for fine-tuning in domain modeling.

Load-bearing premise

The load-bearing premise is that Cosine Similarity between raw generated text and the ground-truth text measures syntactic correctness and BERTScore measures semantic correctness; if those text-overlap scores do not track the actual structure of an Ecore model, the 'optimal' configurations are optimal only for the proxies.

Editorial extensions

If this is right

Tuning can be done once on a single reference model and then reused across domains, avoiding fine-tuning's need for large training datasets and its risk of catastrophic forgetting.
With few-shot prompting, tuned Llama 3.1 settings produce domain models that are statistically significantly better than default settings on most of the ten domains.
With chain-of-thought prompting, tuned settings improve syntactic similarity on nearly all domains, while few-shot is preferable when semantic similarity also matters.
Because the tuned configurations were found for one architecture (Llama 3.1), the results do not automatically extend to other LLMs; the paper proposes testing GPT, BERT, and DeepSeek as future work.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A direct consequence the paper leaves implicit: the search procedure is model- and metric-specific, so the same pipeline could be rerun with any base LLM and any similarity metric, turning hyperparameter tuning into a general pre-processing step for LLM-based generation tasks.
Because all quality judgments use text-overlap proxies, the practical value of the wins depends on whether Cosine Similarity and BERTScore correlate with structural correctness of Ecore models; if they do not, the reported improvements may not survive structural comparison.
The consistent disagreement between Cosine Similarity and BERTScore across domains suggests that a single scalar quality measure may be insufficient for this task, and future optimizers should treat syntactic and semantic fidelity as genuinely separate objectives.
A testable extension: use the same six configurations with a structurally aware metric on the same ten domains; the ranking of configurations would show whether the text-proxy optimum coincides with the model-structure optimum.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

My quick take: this is a reasonably careful empirical study whose central claim is, as written, not supported by its own Table 4. The Abstract and the Answer to RQ2 say that adding hyperparameter tuning to prompt engineering improves quality across nearly all domain models. But the Win/Tie/Loss data show that only the Cosine Similarity (text-overlap) metric improves consistently; BERTScore wins are a minority in CoT for every configuration, and in few-shot only S0 has more BERTScore wins than losses. Per-domain, BERTScore wins exceed losses in 3 of 10 domains under few-shot and 1 of 10 under CoT. So the semantic dimension does not clearly improve, and the paper's own text even acknowledges a 'systematic disagreement' between the metrics. The conclusion then quietly drops the caveat.

What's new and useful: I don't know another paper that tunes Llama 3.1 decoding hyperparameters with NSGA-II plus grid search specifically for Ecore domain model generation, then tests the resulting configurations across ten domains with three prompt styles. The replication package is there, they run 20 repetitions, and they report Wilcoxon and A12 effect sizes. That is a solid empirical skeleton.

The soft spots, in proportion. The metric validity problem is real but not fatal: Cosine and BERTScore on raw text are weak proxies for whether an Ecore model is structurally correct. The paper acknowledges EMFCompare exists but says it has no API for the fitness function. That is a legitimate reason for the tuning loop, but it doesn't excuse using the same proxies as the headline evaluation. A validation subset, comparing generated models with EMFCompare or a structural comparator, would substantially strengthen the claims. Failing that, the conclusions should be restricted to text-similarity outcomes. The RQ1.1 sanity check is in-sample and therefore mostly a fitted-value confirmation; the out-of-sample domains carry the actual weight, and they do show something interesting: tuning alone generalizes poorly, while the combination with prompting helps under Cosine. That is a useful, honest negative-result-adjacent finding. One minor point: no correction for multiple comparisons across the many Wilcoxon tests; worth a sentence in threats to validity.

Who this is for: MDE researchers working with LLM-based modeling assistants, and people doing prompt/hyperparameter optimization for code/model generation. It deserves a serious referee, but the authors need to either fix the abstract and RQ2 answer to match Table 4 or provide structural validation. I'd accept it for review with a major-revision recommendation.

Referee Report

3 major / 6 minor

Summary. The paper proposes using NSGA-II global search followed by grid search to tune five decoding hyperparameters (temperature, top-k, top-p, repetition penalty, max new tokens) of Llama-3.1-8B-Instruct for generating Ecore domain models from textual descriptions. Hyperparameters are tuned on the LIFEMap medical model using Cosine Similarity and BERTScore as fitness functions, producing six Pareto-optimal configurations (S0-S5). The authors evaluate these configurations against default hyperparameters under zero-shot, few-shot, and chain-of-thought prompting across ten domains taken from a prior study, using Win/Tie/Loss counts, Wilcoxon tests, and A12 effect sizes. The paper reports that zero-shot tuning does not generalize, but combining tuning with prompt engineering improves results in most domains, with few-shot claimed to increase both syntactic and semantic similarity.

Significance. The study addresses a practical and underexplored question: whether decoding-hyperparameter tuning can serve as a lightweight alternative to fine-tuning for LLM-based domain modeling. The evaluation design has notable strengths: ten out-of-sample domains, twenty repetitions per configuration, non-parametric statistical testing, effect-size analysis, and a replication package. If the proxy metrics were valid and the headline claim were precisely stated, the results would be useful guidance for MDE practitioners. However, the central claim as written is broader than the data support, and the unvalidated text-overlap metrics leave the actual improvement in modeling quality uncertain. The core out-of-sample evaluation is real, so the issues are fixable, but they require substantive revision rather than copy-editing.

major comments (3)

[Section 5.2, Table 4a] The Answer to RQ2 states that few-shot prompting increases both syntactic and semantic similarity, and the Abstract claims enhancement across nearly all examined domain models, but the BERTScore results in Table 4a do not support the semantic part of this claim. Under RQ2.1 (Few-Shot), only configuration S0 has more BERTScore wins than losses (5/1/4); S1 is 4/0/6, S2 is 4/2/4, S3 is 2/0/8, S4 is 3/0/7, and S5 is 3/0/7. Under RQ2.2 (CoT), no configuration has more BERTScore wins than losses (e.g., S3 is 5/0/5, S0 is 4/0/6, S1 is 1/0/9). The unqualified conclusion should be revised to state that the improvement holds for Cosine Similarity but not for BERTScore, or the authors should provide additional evidence for semantic improvement.
[Sections 3.1 and 4.3] The manuscript treats Cosine Similarity between raw generated text and ground-truth text as a measure of syntactic correctness of an Ecore model, and BERTScore as a measure of semantic correctness, but no evidence is provided that either metric tracks the structural validity or conceptual content of an Ecore metamodel. The authors acknowledge that more specific tools such as EMFCompare exist but were not used because they lack an API. Given that hyperparameters are optimized for these proxies, the reported wins may indicate improved lexical overlap rather than improved domain models. The paper should validate the proxies on a sample, for example by parsing generated Ecore outputs and computing structural agreement with the ground truth, or by manual inspection of a subset of generated models.
[Section 5.1, RQ1.1] The RQ1.1 sanity check uses the LIFEMap model and the same Cosine Similarity and BERTScore metrics that drove the NSGA-II and grid search, so it is a fitted-value confirmation rather than independent evidence of effectiveness. The paper does call this a sanity check, but Section 5.1's statement that the outcome motivates the need for tuning should be tempered by noting that the only out-of-sample tests are RQ1.2 and RQ2. This point is not fatal to the core generalization claim, but the current framing gives RQ1.1 more evidential weight than it can bear.

minor comments (6)

[Section 4.1] The heading "Reseach questions" should be corrected to "Research questions".
[Table 4] The Win/Tie/Loss triples in Table 4a are run together in the manuscript text, making the table very difficult to read; the authors should reformat the table with clear column separators for each RQ and metric.
[Throughout] The model name is inconsistently given as both "Llama 3" and "Llama 3.1"; the paper should use one consistent name (the experimental model appears to be Llama-3.1-8B-Instruct).
[Section 4.2 and Table 4b] The domain name is spelled "CeIO" in Section 4.2 but "CelO" in Table 4b; this should be harmonized.
[Section 3.1] The sentence justifying Cosine Similarity and BERTScore cites previous works [19,39] as related to domain modeling, but those references appear to be metamodel recommender systems; the authors should clarify how those works validate the use of text-overlap metrics as correctness measures for generated metamodels.
[Section 7] Future work mentions experimenting with "GPT or BERT" as additional LLMs for generation; BERT is an encoder model not designed for text generation, so the intended models should be clarified.

Circularity Check

1 steps flagged · score 2.0 of 10

RQ1.1 sanity check is a fitted confirmation on the tuning domain, but the core generalization claim is tested out-of-sample and is not circular.

fitted input called prediction [Section 3.1 (fitness functions) and Section 5.1 (RQ1.1 sanity check)]
"the quality of each individual is evaluated using two fitness functions for syntactical and semantic similarity against a ground truth model ... RQ1.1: Sanity check ... As shown in the plot, the syntactical and semantical quality of domain models generated by the LLM with optimal hyperparameters significantly overcomes the results obtained by the baseline."

The six configurations S0-S5 were produced by NSGA-II plus grid search whose objectives were exactly Cosine Similarity and BERTScore computed against the LIFEMap ground truth. RQ1.1 then evaluates those same configurations on the same LIFEMap model with the same two metrics and reports that they beat the default baseline. This is a check that the optimizer improved its own training objective on the training domain; the improvement is forced by the selection process and provides no independent evidence. The paper itself labels it a sanity check, and the central generalization claims (RQ1.2 and RQ2) are based on ten out-of-sample domains with hyperparameters fixed before evaluation, so this fitted confirmation is not load-bearing for the core claim.

full rationale

The central claim is that hyperparameter tuning combined with prompt engineering improves generated domain models across most of ten examined domains. Those ten domains are not used in the NSGA-II or grid-search tuning: the six Pareto configurations are fixed after tuning on LIFEMap and then applied to the ten out-of-sample domains, so the main evaluation is genuinely out-of-sample. The baseline comparison uses the same prompts and the same statistical tests, and the results are reported per configuration and per domain rather than being selected after the fact. No load-bearing self-citation chain, imported uniqueness theorem, or ansatz-smuggling-via-citation is present; citations to prior work such as the dataset in [13] and metrics in [19,39] are external and non-circular. The only circularity-like step is the RQ1.1 sanity check, which confirms on the same LIFEMap model and the same metrics that the search found improving settings; this is a fitted-value confirmation and is explicitly labeled a sanity check, so it does not support the generalization claim by itself. Separately, the paper's own Table 4 shows that BERTScore wins are a minority in most RQ2 configurations, which undercuts the Abstract's unqualified 'quality' wording; that is an internal-consistency and construct-validity concern rather than a circularity concern.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

All free parameters are decoding hyperparameters fitted to maximize text similarity on LIFEMap. The axioms reflect the main validity assumptions of the empirical design.

free parameters (5)

Temperature = {0.6, 0.8, 1.0, 1.1} across configurations S0-S5
Optimized via NSGA-II and grid search to maximize Cosine Similarity and BERTScore on the LIFEMap domain model.
Top-k = {0, 50}
Part of the reduced search space and final configurations.
Top-p = {0.9, 1.0}
Part of the reduced search space and final configurations.
Max new tokens = {2048, 3072, 4096}
Part of the final configurations; longer token budgets were selected.
Repetition penalty = Not reported in final configurations
Included in the search space (Table 1) and reduced space (Table 2), but omitted from the reported optimal configurations in Table 3, an unreported gap.

assumptions (5)

ad hoc to paper Cosine Similarity and BERTScore computed on raw generated text are valid proxies for syntactic and semantic correctness of Ecore domain models.
Used as fitness functions and evaluation metrics; not validated against structural model comparison, as acknowledged in Section 3.1.
domain assumption The LIFEMap domain model is a correct and representative ground truth for the medical domain.
Serves as the single training target for hyperparameter tuning; no external validation.
ad hoc to paper NSGA-II with population size 30, 10 generations, 90% crossover and 20% mutation sufficiently explores the hyperparameter search space.
Parameters chosen without sensitivity analysis; stated in Section 3.1.
domain assumption The ten domain models from [13] are representative of diverse application domains.
Used to test generalizability; no justification of representativeness beyond citation.
ad hoc to paper The search space ranges in Table 1 (temperature 0.5-2.0, top-k 0-100, etc.) are appropriate for Llama 3.1.
Ranges are based on [4] but are ad hoc for this task.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Investigating the Role of LLMs Hyperparameter Tuning and Prompt Engineering to Support Domain Modeling." pith.science (2026). https://pith.science/paper/D7QQPL3S

@misc{pith2026250714735,
  author       = {Pith},
  title        = {Pith review of: Investigating the Role of LLMs Hyperparameter Tuning and Prompt Engineering to Support Domain Modeling},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/D7QQPL3S}},
  note         = {Machine review of arXiv:2507.14735}
}

read the original abstract

The introduction of large language models (LLMs) has enhanced automation in software engineering tasks, including in Model Driven Engineering (MDE). However, using general-purpose LLMs for domain modeling has its limitations. One approach is to adopt fine-tuned models, but this requires significant computational resources and can lead to issues like catastrophic forgetting. This paper explores how hyperparameter tuning and prompt engineering can improve the accuracy of the Llama 3.1 model for generating domain models from textual descriptions. We use search-based methods to tune hyperparameters for a specific medical data model, resulting in a notable quality improvement over the baseline LLM. We then test the optimized hyperparameters across ten diverse application domains. While the solutions were not universally applicable, we demonstrate that combining hyperparameter tuning with prompt engineering can enhance results across nearly all examined domain models.

Figures

Figures reproduced from arXiv: 2507.14735 by the authors.

**Figure 1.** Excerpt of the LIFEMap domain model the process is still challenging due to the complexity of peculiar domains. Figure 1 depicts an excerpt of the LIFEMap model, which represents a medical Case Report Form (CRF) for patients under specific treatments. In particular, a Patient has a FamilyHistory that may affect the treatment and the corresponding LaboratoryExams. It is worth mentioning that the discussed model is … view at source ↗

**Figure 2.** Hyperparameter tuning methodology [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗

**Figure 3.** RQ1.1: Distribution of Cosine Similarity and BERTScore of models generated by Llama3.1 using the optimal configurations and with standard hyperparameters LIFEMap domain model. Instead, under the BERT Score (i.e., semantical quality), we found no domain models with more wins than losses. In general, this evaluation may lead us to conclude that the optimal solutions returned by our search can not be generalized to d… view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

41 extracted references · 25 canonical work pages

[1]

Ahmad, A., Waseem, M., Liang, P., Fahmideh, M., Aktar, M.S., Mikkonen, T.: Towards Human-Bot Collaborative Software Architecting with ChatGPT. 16 V. Bulhakov et al. In: Proceedings of the 27th International Conference on Evaluation and Assess- ment in Software Engineering. pp. 279–285. EASE ’23, Association for Com- puting Machinery, New York, NY, USA (Ju...

arXiv 2023
[2]

In: Proceedings of the 33rd International Conference on Software Engineering

Arcuri, A., Briand, L.: A practical guide for using statistical tests to assess random- ized algorithms in software engineering. In: Proceedings of the 33rd International Conference on Software Engineering. pp. 1–10. ICSE ’11, Association for Com- puting Machinery, New York, NY, USA (May 2011).https://doi.org/10.1145/ 1985793.1985795, https://dl.acm.org/d...

arXiv
[3]

In: 2024 International Joint Conference on Neural Networks (IJCNN)

Ardimento, P., Bernardi, M.L., Cimitile, M.: Teaching uml using a rag-based llm. In: 2024 International Joint Conference on Neural Networks (IJCNN). pp. 1–8 (2024). https://doi.org/10.1109/IJCNN60899.2024.10651492

arXiv 2024
[4]

Arora, C., Sayeed, A.I., Licorish, S., Wang, F., Treude, C.: Optimizing large lan- guage model hyperparameters for code generation (2024), https://arxiv.org/ abs/2408.10577

arXiv 2024
[5]

In: 2023 ACM/IEEE Interna- tional Conference on Model Driven Engineering Languages and Systems Compan- ion (MODELS-C)

Arulmohan, S., Meurs, M.J., Mosser, S.: Extracting Domain Models from Textual Requirements in the Era of Large Language Models. In: 2023 ACM/IEEE Interna- tional Conference on Model Driven Engineering Languages and Systems Compan- ion (MODELS-C). pp. 580–587. IEEE, Västerås, Sweden (Oct 2023).https:// doi.org/10.1109/MODELS-C59198.2023.00096, https://ieee...

arXiv 2023
[6]

In: Proceedings of the ACM/IEEE 19th International Conference on Model Driven Engineering Languages and Systems

Batot, E., Sahraoui, H.: A generic framework for model-set selection for the unifi- cation of testing and learning mde tasks. In: Proceedings of the ACM/IEEE 19th International Conference on Model Driven Engineering Languages and Systems. p. 374–384. MODELS ’16, Association for Computing Machinery, New York, NY, USA (2016), https://doi.org/10.1145/2976767.2976785

work page arXiv 2016
[8]

Bertram, V., Boß, M., Kusmenko, E., Nachmann, I.H., Rumpe, B., Trotta, D., Wachtmeister, L.: Neural language models and few shot learning for systematic requirements processing in mdse. In: Proceedings of the 15th ACM SIGPLAN InternationalConferenceonSoftwareLanguageEngineering.p.260–265.SLE2022, Association for Computing Machinery, New York, NY, USA (202...

work page arXiv 2022
[9]

Bulhakov, V., d’Aloisio, G., Di Sipio, C., Di Marco, A., Di Ruscio, D.: LLama Ecore Study (Jan 2025),https://github.com/VPLEV23/ER_LLM

work page 2025

Show all 41 references

[10]

ACM Trans

Burgueño, L., Di Ruscio, D., Sahraoui, H., Wimmer, M.: Automation in model- driven engineering: A look back, and ahead. ACM Trans. Softw. Eng. Methodol. (Jan 2025), https://doi.org/10.1145/3712008, just Accepted

2025 doi
[11]

Software and Systems Modeling 22(3), 781–793 (Jun 2023)

Cámara, J., Troya, J., Burgueño, L., Vallecillo, A.: On the assessment of generative AI in modeling tasks: an experience report with ChatGPT and UML. Software and Systems Modeling 22(3), 781–793 (Jun 2023). https://doi.org/10.1007/ s10270-023-01105-5, https://doi.org/10.1007/s...

2023 doi
[12]

In: Proceedings of the 45th Inter- national Conference on Software Engineering: New Ideas and Emerging Results

Chaaben, M.B., Burgueño, L., Sahraoui, H.: Towards Using Few-Shot Prompt Learning for Automating Model Completion. In: Proceedings of the 45th Inter- national Conference on Software Engineering: New Ideas and Emerging Results. pp. 7–12. ICSE-NIER ’23, IEEE, IEEE Press, Melbour...

2023
[13]

In: 2023 ACM/IEEE 26th International Conference on Model Driven En- gineering Languages and Systems (MODELS)

Chen, K., Yang, Y., Chen, B., Hernández López, J.A., Mussbacher, G., et al.: Automated Domain Modeling with Large Language Models: A Comparative Study. In: 2023 ACM/IEEE 26th International Conference on Model Driven En- gineering Languages and Systems (MODELS). pp. 162–172 (Oc...

2023
[14]

In: Meila, M., Zhang, T

Chen, P.H., Wei, W., Hsieh, C.J., Dai, B.: Overcoming catastrophic forgetting by bayesian generative regularization. In: Meila, M., Zhang, T. (eds.) Proceedings of the 38th International Conference on Machine Learning. Proceedings of Machine Learning Research, vol. 139, pp. 17...

2021
[15]

Costa, C.D., López, J.A.H., Cuadrado, J.S.: Modelmate: A recommender for tex- tual modeling languages based on pre-trained language models. In: Proceedings of the ACM/IEEE 27th International Conference on Model Driven Engineering Lan- guagesandSystems.p.183–194.MODELS’24,Assoc...

2024
[16]

In: Proceedings of the 18th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement

d’Aloisio, G., Fortz, S., Hanna, C., Fortunato, D., Bensoussan, A., Mendiluze Usan- dizaga, E., Sarro, F.: Exploring llm-driven explanations for quantum algorithms. In: Proceedings of the 18th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement. ...

2024
[17]

IEEE Trans

Deb, K., Agrawal, S., Pratap, A., Meyarivan, T.: A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Trans. Evol. Comput.6(2), 182–197 (2002)

2002
[18]

arXiv preprint arXiv:2409.19027 (2024)

Deroy, A., Maity, S.: Code generation and algorithmic problem solving using llama 3.1 405b. arXiv preprint arXiv:2409.19027 (2024)

2024 arXiv
[19]

Software and Systems Modeling pp

Di Rocco, J., Di Ruscio, D., Di Sipio, C., Nguyen, P.T., Pierantonio, A.: Memorec: a recommender system for assisting modelers in specifying metamodels. Software and Systems Modeling pp. 1–21 (2022)

2022
[20]

In: Proceedings of the ACM/IEEE 27th International Conference on Model Driven Engineering Languages and Systems

Di Sipio, C., Rubei, R., Di Rocco, J., Di Ruscio, D., Iovino, L.: On the use of llms to support the development of domain-specific modeling languages. In: Proceedings of the ACM/IEEE 27th International Conference on Model Driven Engineering Languages and Systems. p. 596–601. M...

2024
[21]

ACM Trans

Fan, L., Liu, J., Liu, Z., Lo, D., Xia, X., Li, S.: Exploring the capabilities of llms for code change related tasks. ACM Trans. Softw. Eng. Methodol. (Dec 2024).https: //doi.org/10.1145/3709358, https://doi.org/10.1145/3709358, just Accepted

2024 doi
[22]

ACM computing surveys (CSUR)28(1), 77–80 (1996)

Forrest, S.: Genetic algorithms. ACM computing surveys (CSUR)28(1), 77–80 (1996)

1996
[23]

Evolu- tionary Computation 22(4), 651–678 (2014).https://doi.org/10.1162/EVCO_a_ 00128

Giagkiozis, I., Fleming, P.J.: Pareto front estimation for decision making. Evolu- tionary Computation 22(4), 651–678 (2014).https://doi.org/10.1162/EVCO_a_ 00128

2014 doi
[24]

Gong, J., Li, S., d’Aloisio, G., Ding, Z., Ye, Y., Langdon, W.B., Sarro, F.: Greensta- bleyolo: Optimizing inference time and image quality of text-to-image generation. 18 V. Bulhakov et al. In: International Symposium on Search Based Software Engineering. pp. 70–76. Springer ...

2024
[25]

In: Adaptive and Natural Computing Algorithms: 8th Interna- tional Conference, ICANNGA 2007, Warsaw, Poland, April 11-14, 2007, Proceed- ings, Part I 8

Hamdani, T.M., Won, J.M., Alimi, A.M., Karray, F.: Multi-objective feature selec- tion with nsga ii. In: Adaptive and Natural Computing Algorithms: 8th Interna- tional Conference, ICANNGA 2007, Warsaw, Poland, April 11-14, 2007, Proceed- ings, Part I 8. pp. 240–247. Springer (2007)

2007
[26]

Applied Soft Computing133, 109916 (Jan 2023)

Hort, M., Moussa, R., Sarro, F.: Multi-objective search for gender-fair and seman- tically correct word embeddings. Applied Soft Computing133, 109916 (Jan 2023). https://doi.org/10.1016/j.asoc.2022.109916, https://www.sciencedirect. com/science/article/pii/S1568494622009656

2023
[27]

In- ternational Journal of Computer Applications162(10) (2017)

Kora, P., Yadlapalli, P.: Crossover operators in genetic algorithms: A review. In- ternational Journal of Computer Applications162(10) (2017)

2017
[28]

In: Proceedings of the 34th Interna- tional Conference on Neural Information Processing Systems

Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Yih, W.t., Rocktäschel, T., Riedel, S., Kiela, D.: Retrieval-augmented generation for knowledge-intensive nlp tasks. In: Proceedings of the 34th Interna- tional Conference on Neura...

2020
[29]

Software and Systems Modeling pp

López, J.A.H., Cánovas Izquierdo, J.L., Cuadrado, J.S.: Modelset: a dataset for machine learning in model-driven engineering. Software and Systems Modeling pp. 1–20 (2021)

2021
[30]

In: Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval

Ma, X., Wang, L., Yang, N., Wei, F., Lin, J.: Fine-tuning llama for multi-stage text retrieval. In: Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval. pp. 2421–2425 (2024)

2024
[31]

meta.com/llama3/

Meta: Meta LLama3.https://llama.meta.com/llama3/ (2024), https://llama. meta.com/llama3/

2024
[32]

Software and Systems Modeling19(5), 1045– 1053 (Sep 2020).https://doi.org/10.1007/s10270-020-00814-5, http://link

Mussbacher, G., Combemale, B., Kienzle, J., Abrahão, S., Ali, H., Bencomo, N., Búr, M., Burgueño, L., Engels, G., Jeanjean, P., Jézéquel, J.M., Kühn, T., Mosser, S., Sahraoui, H., Syriani, E., Varró, D., Weyssow, M.: Opportunities in intelligent modeling assistance. Software a...

2020 doi
[33]

Pearson Higher Education (2004)

Rumbaugh, J., Jacobson, I., Booch, G.: Unified Modeling Language Reference Manual, The (2nd Edition). Pearson Higher Education (2004)

2004
[34]

In: Proceedings of the 23rd ACM/IEEE International Conference on Model Driven Engineering Languages and Systems: Companion Proceedings

Saini, R., Mussbacher, G., Guo, J.L.C., Kienzle, J.: Domobot: A bot for auto- mated and interactive domain modelling. In: Proceedings of the 23rd ACM/IEEE International Conference on Model Driven Engineering Languages and Systems: Companion Proceedings. MODELS ’20, Association...

2020
[35]

Information processing & management24(5), 513–523 (1988)

Salton, G., Buckley, C.: Term-weighting approaches in automatic text retrieval. Information processing & management24(5), 513–523 (1988)

1988
[36]

In: Proceedings of the 38th International Conference on Software Engineering

Sarro, F., Petrozziello, A., Harman, M.: Multi-objective software effort estimation. In: Proceedings of the 38th International Conference on Software Engineering. pp. 619–630. ICSE ’16, Association for Computing Machinery, New York, NY, USA (May 2016).https://doi.org/10.1145/2...

2016
[37]

Journal of Educational and Behavioral Statistics 25(2), 101–132 (2000)

Vargha, A., Delaney, H.D.: A critique and improvement of the cl common language effect size statistics of mcgraw and wong. Journal of Educational and Behavioral Statistics 25(2), 101–132 (2000)

2000
[38]

Advances in neural information processing systems35, 24824–24837 (2022) Investigating LLMs Tuning and Prompt Engineering for Domain Modeling 19

Wei, J., Wang, X., Schuurmans, D., Bosma, M., Xia, F., Chi, E., Le, Q.V., Zhou, D., et al.: Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems35, 24824–24837 (2022) Investigating LLMs Tuning and Prompt Engin...

2022
[39]

Software and Systems Modeling pp

Weyssow, M., Sahraoui, H., Syriani, E.: Recommending metamodel concepts dur- ing modeling activities with pre-trained language models. Software and Systems Modeling pp. 1–19 (2022)

2022
[40]

Springer Berlin Heidelberg, Berlin, Heidelberg (2012)

Wohlin, C., Runeson, P., Höst, M., Ohlsson, M.C., Regnell, B., Wesslén, A.: Exper- imentation in Software Engineering. Springer Berlin Heidelberg, Berlin, Heidelberg (2012). https://doi.org/10.1007/978-3-642-29044-2 , http://link.springer. com/10.1007/978-3-642-29044-2

2012 doi
[41]

Encyclopedia of Biostatistics8 (2005)

Woolson, R.F.: Wilcoxon signed-rank test. Encyclopedia of Biostatistics8 (2005)

2005
[42]

arXiv preprint arXiv:1904.09675 (2019)

Zhang, T., Kishore, V., Wu, F., et al.: Bertscore: Evaluating text generation with bert. arXiv preprint arXiv:1904.09675 (2019)

2019 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.

[1] [1]

Ahmad, A., Waseem, M., Liang, P., Fahmideh, M., Aktar, M.S., Mikkonen, T.: Towards Human-Bot Collaborative Software Architecting with ChatGPT. 16 V. Bulhakov et al. In: Proceedings of the 27th International Conference on Evaluation and Assess- ment in Software Engineering. pp. 279–285. EASE ’23, Association for Com- puting Machinery, New York, NY, USA (Ju...

arXiv 2023

[2] [2]

In: Proceedings of the 33rd International Conference on Software Engineering

Arcuri, A., Briand, L.: A practical guide for using statistical tests to assess random- ized algorithms in software engineering. In: Proceedings of the 33rd International Conference on Software Engineering. pp. 1–10. ICSE ’11, Association for Com- puting Machinery, New York, NY, USA (May 2011).https://doi.org/10.1145/ 1985793.1985795, https://dl.acm.org/d...

arXiv

[3] [3]

In: 2024 International Joint Conference on Neural Networks (IJCNN)

Ardimento, P., Bernardi, M.L., Cimitile, M.: Teaching uml using a rag-based llm. In: 2024 International Joint Conference on Neural Networks (IJCNN). pp. 1–8 (2024). https://doi.org/10.1109/IJCNN60899.2024.10651492

arXiv 2024

[4] [4]

Arora, C., Sayeed, A.I., Licorish, S., Wang, F., Treude, C.: Optimizing large lan- guage model hyperparameters for code generation (2024), https://arxiv.org/ abs/2408.10577

arXiv 2024

[5] [5]

In: 2023 ACM/IEEE Interna- tional Conference on Model Driven Engineering Languages and Systems Compan- ion (MODELS-C)

Arulmohan, S., Meurs, M.J., Mosser, S.: Extracting Domain Models from Textual Requirements in the Era of Large Language Models. In: 2023 ACM/IEEE Interna- tional Conference on Model Driven Engineering Languages and Systems Compan- ion (MODELS-C). pp. 580–587. IEEE, Västerås, Sweden (Oct 2023).https:// doi.org/10.1109/MODELS-C59198.2023.00096, https://ieee...

arXiv 2023

[6] [6]

In: Proceedings of the ACM/IEEE 19th International Conference on Model Driven Engineering Languages and Systems

Batot, E., Sahraoui, H.: A generic framework for model-set selection for the unifi- cation of testing and learning mde tasks. In: Proceedings of the ACM/IEEE 19th International Conference on Model Driven Engineering Languages and Systems. p. 374–384. MODELS ’16, Association for Computing Machinery, New York, NY, USA (2016), https://doi.org/10.1145/2976767.2976785

work page arXiv 2016

[7] [8]

Bertram, V., Boß, M., Kusmenko, E., Nachmann, I.H., Rumpe, B., Trotta, D., Wachtmeister, L.: Neural language models and few shot learning for systematic requirements processing in mdse. In: Proceedings of the 15th ACM SIGPLAN InternationalConferenceonSoftwareLanguageEngineering.p.260–265.SLE2022, Association for Computing Machinery, New York, NY, USA (202...

work page arXiv 2022

[8] [9]

Bulhakov, V., d’Aloisio, G., Di Sipio, C., Di Marco, A., Di Ruscio, D.: LLama Ecore Study (Jan 2025),https://github.com/VPLEV23/ER_LLM

work page 2025