An evolutionary algorithm for mining rare association rules: a Big Data approach

Association rule mining is one of the most well-known techniques to discover interesting relations between items in data. To date, this task has been mainly focused on the discovery of frequent relationships. However, it is often interesting to focus on those that do not occur frequently. Rare association rule mining is an alluring field aiming at describing rare cases or unexpected behavior. This field is really useful over Big Data where abnormal endeavor are more curious than common behavior. In this sense, our aim is to propose a new evolutionary algorithm based on grammars to obtain rare association rules on Big Data. The novelty of our work is that it is eminently designed to be parallel, enabling its use over emerging technologies as Spark and Flink. Furthermore, while other algorithms focus on maximizing a couple of quality measure ignoring the rest, our fitness function has been precisely designed to obtain a trade-off while maximizing a set of well-known quality measures. The experimental study includes more than 70 datasets revealing alluring results in efficiency when more than 300 million of instances and file sizes up to 250 GBytes are considered, and proving that it is able to run efficiently in huge volumes of data.