A Survey on Evolutionary Construction of Deep Neural Networks

Research output: Journal Publications and Reviews (RGC: 21, 22, 62)21_Publication in refereed journalpeer-review

View graph of relations

Author(s)

  • Xun Zhou
  • A. K. Qin
  • Maoguo Gong
  • Kay Chen Tan

Related Research Unit(s)

Detail(s)

Original languageEnglish
Journal / PublicationIEEE Transactions on Evolutionary Computation
Online published13 May 2021
Publication statusOnline published - 13 May 2021

Abstract

Automated construction of deep neural networks (DNNs) has become a research hot spot nowadays because DNN’s performance is heavily influenced by its architecture and parameters which are highly task-dependent, but it is notoriously difficult to find the most appropriate DNN in terms of architecture and parameters to best solve a given task. In this work, we provide an insight into the automated DNN construction process by formulating it into a multi-level multi-objective large-scale optimization problem with constraints, where the non-convex, non-differentiable and black-box nature of this problem makes evolutionary algorithms (EAs) to stand out as a promising solver. Then, we give a systematical review of existing evolutionary DNN construction techniques from different aspects of this optimization problem and analyze the pros and cons of using EA-based methods in each aspect. This work aims to help DNN researchers to better understand why, where, and how to utilize EAs for automated DNN construction and meanwhile help EA researchers to better understand the task of automated DNN construction so that they may focus more on EA-favored optimization scenarios to devise more effective techniques.

Research Area(s)

  • Computational modeling, Computer architecture, Data models, Mathematical model, Optimization, Search problems, Task analysis