Sets of receiver operating characteristic curves and their use in the evaluation of multi-class classification

Within the last two decades, Receiver Operating Characteristic (ROC) Curves have become a standard tool for the analysis and comparison of classifiers since they provide a convenient graphical display of the trade-off between true and false positive classification rates for two class problems. However, there has been relatively little work examining ROC for more than two classes. Here we present an extension of ROC curves which can be used for illustrating and analyzing the quality of multi-class classifiers. Instead of using one single curve, we deal with sets of curves which are calculated for each class separately. These are used for analyzing not only how exactly the classes are separated, but also how clearly the classifier is able to distinguish the given classes. Apart from making it possible to analyze the results graphically, several values describing the classifier's quality can be calculated.