Generating counterexamples of model-based software product lines

In a model-based software product line (MSPL), the variability of the domain is characterized in a variability model and the core artifacts are base models conforming to a modeling language (also called metamodel). A realization model connects the features of the variability model to the base model elements, triggering operations over these elements based on a configuration. The design space of an MSPL is extremely complex to manage for the engineer, since the number of variants may be exponential and the derived product models have to be conforming to numerous well-formedness and business rules. In this paper, the objective is to provide a way to generate MSPLs, called counterexamples (also called antipatterns), that can produce invalid product models despite a valid configuration in the variability model. We describe the foundations and motivate the usefulness of counterexamples (e.g., inference of guidelines or domain-specific rules to avoid earlier the specification of incorrect mappings; testing oracles for increasing the robustness of derivation engines given a modeling language). We provide a generic process, based on the common variability language (CVL) to randomly search the space of MSPLs for a specific modeling language. We develop LineGen a tool on top of CVL and modeling technologies to support the methodology and the process. LineGen targets different scenarios and is flexible to work either with just a domain metamodel as input or also with pre-defined variability models and base models. We validate the effectiveness of this process for three formalisms at different scales (up to 247 metaclasses and 684 rules). We also apply the approach in the context of a real industrial scenario involving a large-scale metamodel.