A language-parametric test coverage framework for executable domain-specific languages


Test coverage is an effective technique to measure test case quality and to enable fault localization. However, for Executable Domain-Specific Languages (xDSLs), coverage metrics and associated tools are currently manually defined for each xDSL resulting in costly, error-prone, and non-reusable work. To address this problem, we propose a novel language-parametric test coverage framework for xDSLs. We define two coverage metrics adapted to xDSLs: model element and branch coverage. For performing coverage measurements, we propose a generic technique which can be used out-of-the-box by domain experts using any xDSL to define, execute, and test models. In addition, the coverage of model elements and branches can be parameterized for a given xDSL through the definition of coverage rules using a dedicated language. We showcase two test coverage applications for xDSLs: measuring the quality of test suites for executable models and localizing the models’ faults using Spectrum-Based Fault Localization techniques. We evaluate our approach using four different xDSLs. Results show that (i) we can generate meaningful coverage measurements for all investigated models, (ii) the provided coverage rule language enables framework parameterization for all xDSLs, and (iii) the computed coverage measurements are useful in identifying defects of the models.

Journal of Systems & Software
Faezeh Khorram
Faezeh Khorram
Senior Research Engineer | PhD

My research interests include Model-Based Verification and Validation, Model-Driven Engineering (MDE), Domain-Specific Languages (DSL), Language Engineering, Debugging and Testing of models.