The Difference Between the Accuracy of Real and the Corresponding Random Model is a Useful Parameter for Validation of Two-State Classification Model Quality

The simplest and the most commonly used measure for assess the classification model quality is parameter Q(2) = 100 (p + n) / N (%) named the classification accuracy, p, n and N are the total numbers of correctly predicted compounds in the first and in the second class, and the total number of elements of classes (compounds) in data set, respectively. Moreover, the most probable accuracy that can be obtained by a random model is calculated for two-state model by the formulae Q(2,rnd) = 100 [(p + u) (p + o) + (n + u) (n + o)] / N-2 (%), where u and o are thetotal number of under-predictions (when class 1 is predicted by the model as class 2) and over-predictions (when class 2 is predicted by the model as class 1) in data set, respectively. Finally, the difference between these two parameter Delta Q(2) = Q(2) - Q(2), rnd is introduced, and it is suggested to compute and give Delta Q(2) for each two-state classification model to assess its contribution over the accuracy of the corresponding random model. When data set is ideally balanced having the same numbers of elements in both classes, the two-state classification problem is the most difficult with maximal Q(2) = 100 % and Q(2,rnd) = 50 %, giving the maximal Q(2) = 50 %. The usefulness of Q(2) parameter is illustrated in comparative analysis on two-class classification models from literature for prediction of secondary structure of membrane proteins and on several quantitative structure-property models. Real contributions of these models over the random level of accuracy is calculated, and their Delta Q(2) values are compared mutually and with the value of.Q(2) (= 50 %) for the most difficult two-state classification model.