Training error and sensitivity-based ensemble feature selection

Ensemble feature selection combines feature selection and ensemble learning to improve the generalization capability of ensemble systems. However, current methods minimizing only the training error may not generalize well on future unseen samples. In this paper, we propose a training error and sensitivity-based ensemble feature selection method.The NSGA-III is applied to find optimal feature subsets by minimizing two objective functions of the whole ensemble system simultaneously: the training error and the sensitivity of the ensemble. With this scheme, the ensemble system maintains both high accuracy and high stability which is expected to achieve a high generalization capability. Experimental results on 18 datasets show that the proposed method significantly outperforms state-of-the-art methods.