A novel mutual information-based feature selection algorithm

Cassarà P., Rozza A.

Feature selection  Mutual information  Markov blanket  Cross-entropy  Numerical analysis  Optimization  Pattern recognition 

From a machine learning point of view to identify a subset of relevant features from a real data set can be useful to improve the results achieved by classification methods and to reduce their time and space complexity. To achieve this goal, feature selection methods are usually employed. These approaches assume that the data contains redundant or irrelevant attributes that can be eliminated. In this work we propose a novel feature selection technique that exploits Mutual Information and that is able to automatically estimates the number of dimensions to retain. The main advantages of this new approach are: the ability to automatically estimate the number of features to retain, and the possibility to rank the features to select from the most probable to the less probable. Experiments on standard real data sets and the comparison with state-of-the-art feature selection techniques confirms the high quality of our approach.

Source: ISTI Technical reports, 2015