Effective feature selection scheme using mutual information

This article proposes a novel mutual information-based feature selection scheme. In this scheme, the mutual information is estimated directly in an effective way even when one is handling a relative small data set. At the same time, the computation efficiency of the mutual information estimation is improved by proposing a supervised data compression algorithm. With these contributions, the proposed feature selection scheme is able to effectively identify the salience features. The proposed methodology is compared with the related study through applying to different classification problems in which the number of features ranged from less than 10 to over 12,600. The presented results are very promising and corroborate the contributions of the proposed methodology. © 2004 Elsevier B.V. All rights reserved.