RGP : Neural Network Pruning Through Regular Graph With Edges Swapping

Research output: Journal Publications and ReviewsRGC 21 - Publication in refereed journalpeer-review

3 Scopus Citations
View graph of relations

Author(s)

  • Zhuangzhi Chen
  • Jingyang Xiang
  • Yao Lu
  • Qi Xuan
  • Zhen Wang
  • Xiaoniu Yang

Related Research Unit(s)

Detail(s)

Original languageEnglish
Journal / PublicationIEEE Transactions on Neural Networks and Learning Systems
Online published13 Jun 2023
Publication statusOnline published - 13 Jun 2023

Abstract

Deep learning technology has found a promising application in lightweight model design, for which pruning is an effective means of achieving a large reduction in both model parameters and float points operations (FLOPs). The existing neural network pruning methods mostly start from the consideration of the importance of model parameters and design parameter evaluation metrics to perform parameter pruning iteratively. These methods were not studied from the perspective of network model topology, so they might be effective but not efficient, and they require completely different pruning for different datasets. In this article, we study the graph structure of the neural network and propose a regular graph pruning (RGP) method to perform a one-shot neural network pruning. Specifically, we first generate a regular graph and set its node-degree values to meet the preset pruning ratio. Then, we reduce the average shortest path-length (ASPL) of the graph by swapping edges to obtain the optimal edge distribution. Finally, we map the obtained graph to a neural network structure to realize pruning. Our experiments demonstrate that the ASPL of the graph is negatively correlated with the classification accuracy of the neural network and that RGP has a strong precision retention capability with high parameter reduction (more than 90%) and FLOPs reduction (more than 90%) (the code for quick use and reproduction is available at https://github.com/Holidays1999/Neural-Network-Pruning-through-its-RegularGraph-Structure). © 2023 IEEE.

Research Area(s)

  • Convolutional neural network, deep learning, network architecture optimization, neural network pruning

Citation Format(s)

RGP: Neural Network Pruning Through Regular Graph With Edges Swapping. / Chen, Zhuangzhi; Xiang, Jingyang; Lu, Yao et al.
In: IEEE Transactions on Neural Networks and Learning Systems, 13.06.2023.

Research output: Journal Publications and ReviewsRGC 21 - Publication in refereed journalpeer-review