Feature Selection Based on a New Dependency Measure

Feature selection is a process commonly used in machine learning, wherein a subset of the features available from the data are selected for application of a learning algorithm. Feature selection is effective in reducing dimensionality, removing irrelevant data, increasing learning accuracy and efficiency. In this paper we propose a new information distance to measure the relevancy of two features. Unlike the information measure in previous feature selection works, our proposed information distance meets the condition of triangle inequality. We use InfoDist to feature selection and the experimental results showed it has a better performance.