ConCave-Convex procedure for support vector machines with Huber loss for text classification

The classical support vector machine (SVM) adopts the linear Hinge loss whereas the least squares SVM (LS-SVM) employs the quadratically growing least squares loss function. The robust Ramp loss function is employed in Ramp loss SVM (RSVM) that truncates the Hinge loss function and becomes flat a specified point afterwards, thus, increases robustness to outliers. Recently proposed SVM with pinball loss (pin-SVM) utilizes pinball loss function that maximizes the margin between the class hyperplanes based on quantile distance. Huber loss function is the generalization of linear Hinge loss and quadratic loss. Huber loss solves sensitivity issues of least squares loss to noise and outlier. In this work, we employ the robust Huber loss function for SVM classification for improved generalization performance. The cost function of the proposed approach consists of one convex and one non-convex part, which might sometimes provide local optimum solution instead of a global optimum. We suggest a ConCave-Convex Procedure (CCCP) to resolve this issue. Additionally, the proximal cost is scaled for each class sample based on their class size to reduce the effect of the class imbalance problem. Thus, it can be claimed that the proposed approach incorporates class imbalance learning as well. Extensive experimental analysis establishes efficacy of the proposed method. Furthermore, a sequential minimal optimization (SMO) procedure for high dimensional HSVM is proposed and its performance is tested on two text classification datasets.