Prompt Engineering for Advancing Software Engineering: Using Generative Language Models in the Development of Domain-Specific Languages

Abstract

In this research, a comprehensive framework employing large language models (LLMs) and prompt engineering techniques is developed with the aim of optimizing the creation of domain-specific languages (DSLs). Domain experts utilize DSLs to create precise, high-level specifications that can be customized to their application domains. However, challenges regarding usability, semantic correctness, and syntax design are often encountered in the development process. With recent advances in LLaMA, Claude, DeepSeek, Gemini, and ChatGPT, it is now possible to automate and refine several DSL development stages through carefully written prompts. The proposed framework provides structured prompt patterns that are customized to each successive stage of the DSL life cycle: domain knowledge acquisition, grammar definition, syntax construction, and validation. A case study involving a hospital monitoring system is used to explore the effectiveness of the models and the prompting strategies. This study also classifies the prompting techniques and assesses five LLMs in terms of readability, expressiveness, accuracy, syntax clarity, and completeness. The findings suggest that prompt engineering significantly contributes to the development of DSLs, with the five LLMs excelling in different evaluation measurements. For example, DeepSeek and LLaMA display superior syntactic precision, while Claude and ChatGPT perform better in terms of expressiveness and readability. The need to align the choice of LLM with stakeholder expectations, whether they be technical or domain specific, is highlighted by expert assessments. This research contributes to the body of knowledge relating to LLM-assisted DSL development and provides a basis for further study in adaptive prompting and multimodal integration.