Surrogate-assisted and filter-based multi-objective evolutionary feature selection for deep learning

Abstract

Feature selection for deep learning prediction mod- els is a difficult topic for researchers to tackle. Most of the ap- proaches proposed in the literature consist of embedded methods through the use of hidden layers added to the neural network architecture that modify the weights of the units associated with each input attribute so that the worst attributes have less weight in the learning process. Other approaches used for deep learning are filter methods, which are independent of the learning algorithm, which can limit the precision of the prediction model. Wrapper methods are impractical with deep learning due to their high computational cost. In this paper, we propose new attribute subset evaluation feature selection methods for deep learning of the wrapper, filter and wrapper-filter hybrid types, where multi-objective and many-objective evolutionary algorithms are used as search strategies. A novel surrogate-assisted approach is used to reduce the high computational cost of the wrapper-type objective function, while the filter-type objective functions are based on correlation and an adaptation of the reliefF algorithm. The proposed techniques have been applied in a time series forecasting problem of air quality in the Spanish south-east and an indoor temperature forecasting problem in a domotic house, with promising results compared to other feature se