Fine-tuning 7B code LLMs on a custom multi-file DSL dataset achieves structural fidelity of 1.00, high exact-match accuracy, and practical utility validated by expert survey and execution checks.
and Methani, C
5 Pith papers cite this work, alongside 1 external citations. Polarity classification is still indexing.
5
Pith papers citing it
1
external citations · Pith
abstract
Natural Language to Code Generation has made significant progress in recent years with the advent of Large Language Models(LLMs). While generation for general-purpose languages like C, C++, and Python has improved significantly, LLMs struggle with custom function names in Domain Specific Languages or DSLs. This leads to higher hallucination rates and syntax errors, specially for DSLs having a high number of custom function names. Additionally, constant updates to function names add to the challenge as LLMs need to stay up-to-date. In this paper, we present optimizations for using Retrieval Augmented Generation (or RAG) with LLMs for DSL generation along with an ablation study comparing these strategies. We generated a train as well as test dataset with a DSL to represent automation tasks across roughly 700 APIs in public domain. We used the training dataset to fine-tune a Codex model for this DSL. Our results showed that the fine-tuned model scored the best on code similarity metric. With our RAG optimizations, we achieved parity for similarity metric. The compilation rate, however, showed that both the models still got the syntax wrong many times, with RAG-based method being 2 pts better. Conversely, hallucination rate for RAG model lagged by 1 pt for API names and by 2 pts for API parameter keys. We conclude that an optimized RAG model can match the quality of fine-tuned models and offer advantages for new, unseen APIs.
citation-role summary
background 3
citation-polarity summary
years
2026 5roles
background 3polarities
background 3representative citing papers
PrismaDV generates task-aware data unit tests by jointly analyzing downstream code and dataset profiles, outperforming task-agnostic baselines on new benchmarks spanning 60 tasks, with SIFTA enabling automatic prompt optimization that beats hand-written prompts.
A conceptual model proposes representing LLM workflow definitions, instances, and inference records as persistent typed objects in a shared knowledge substrate, distinguishing deterministic derive from LLM-mediated infer.
Few-shot prompting improves syntactic validity of LLM-generated code across ATL, ETL, QVTo, and Reactions, but semantic correctness gains remain uneven and language-dependent.
Fine-tuned small language models outperform larger models in natural language to domain-specific code generation with improved performance, latency, and the ability to adapt to customer-specific scenarios without losing general capabilities.
citing papers explorer
-
Leveraging LLMs for Multi-File DSL Code Generation: An Industrial Case Study
Fine-tuning 7B code LLMs on a custom multi-file DSL dataset achieves structural fidelity of 1.00, high exact-match accuracy, and practical utility validated by expert survey and execution checks.
-
PrismaDV: Automated Task-Aware Data Unit Test Generation
PrismaDV generates task-aware data unit tests by jointly analyzing downstream code and dataset profiles, outperforming task-agnostic baselines on new benchmarks spanning 60 tasks, with SIFTA enabling automatic prompt optimization that beats hand-written prompts.
-
Workflow as Knowledge: Semantic Persistence for LLM-Mediated Workflows
A conceptual model proposes representing LLM workflow definitions, instances, and inference records as persistent typed objects in a shared knowledge substrate, distinguishing deterministic derive from LLM-mediated infer.
-
LLM4MTLs: Automated Generation and Empirical Evaluation of Model Transformation Languages
Few-shot prompting improves syntactic validity of LLM-generated code across ATL, ETL, QVTo, and Reactions, but semantic correctness gains remain uneven and language-dependent.
-
SLM Finetuning for Natural Language to Domain Specific Code Generation in Production
Fine-tuned small language models outperform larger models in natural language to domain-specific code generation with improved performance, latency, and the ability to adapt to customer-specific scenarios without losing general capabilities.