Modeling Label Interactions in Multi-label Classification: A Multi-structure SVM Perspective

Multi-label classification has attracted much interest due to its wide applicability. Modeling label interactions and investigating their impact on classifier quality are crucial aspects of multi-label classification. In this paper, we propose a multi-structure SVM (called MSSVM) which allows the user to hypothesize multiple label interaction structures and helps to identify their importance in improving generalization performance. We design an efficient optimization algorithm to solve the proposed MSSVM. Extensive empirical evaluation provides fresh and interesting insights into the following questions: (a) How do label interactions affect multiple performance metrics typically used in multi-label classification? (b) Do higher order label interactions significantly impact a given performance metric for a particular dataset? (c) Can we make useful suggestions on the label interaction structure? and (d) Is it always beneficial to model label interactions in multi-label classification?