Realization of Spatial Sparseness by Deep ReLU Nets with Massive Data
Research output: Journal Publications and Reviews › RGC 21 - Publication in refereed journal › peer-review
Author(s)
Related Research Unit(s)
Detail(s)
Original language | English |
---|---|
Pages (from-to) | 229-243 |
Journal / Publication | IEEE Transactions on Neural Networks and Learning Systems |
Volume | 33 |
Issue number | 1 |
Online published | 16 Oct 2020 |
Publication status | Published - Jan 2022 |
Link(s)
Abstract
The great success of deep learning poses urgent challenges for understanding its working mechanism and rationality. The depth, structure, and massive size of the data are recognized to be three key ingredients for deep learning. Most of the recent theoretical studies for deep learning focus on the necessity and advantages of depth and structures of neural networks. In this article, we aim at rigorous verification of the importance of massive data in embodying the outperformance of deep learning. In particular, we prove that the massiveness of data is necessary for realizing the spatial sparseness, and deep nets are crucial tools to make full use of massive data in such an application. All these findings present the reasons why deep learning achieves great success in the era of big data though deep nets and numerous network structures have been proposed at least 20 years ago.
Research Area(s)
- Deep nets, learning theory, massive data, spatial sparseness
Citation Format(s)
Realization of Spatial Sparseness by Deep ReLU Nets with Massive Data. / Chui, Charles K.; Lin, Shao-Bo; Zhang, Bo et al.
In: IEEE Transactions on Neural Networks and Learning Systems, Vol. 33, No. 1, 01.2022, p. 229-243.
In: IEEE Transactions on Neural Networks and Learning Systems, Vol. 33, No. 1, 01.2022, p. 229-243.
Research output: Journal Publications and Reviews › RGC 21 - Publication in refereed journal › peer-review