Affiliations: [a] School of Computer and Communication Engineering, Changsha University of Science and Technology, Hunan, China | [b] Hunan Provincial Key Laboratory of Intelligent Processing of Big Data on Transportation, Hunan, China
Abstract: Pruning of neural networks is undoubtedly a popular approach to cope with the current compression of large-scale, high-cost network models. However, most of the existing methods require a high level of human-regulated pruning criteria, which requires a lot of human effort to figure out a reasonable pruning strength. One of the main reasons is that there are different levels of sensitivity distribution in the network. Our main goal is to discover compression methods that adapt to this distribution to avoid deep architectural damage to the network due to unnecessary pruning. In this paper, we propose a filter texture distribution that affects the training of the network. We also analyze the sensitivity of each of the diverse states of this distribution. To do so, we first use a multidimensional penalty method that can analyze the potential sensitivity based on this texture distribution to obtain a pruning-friendly sparse environment. Then, we set up a lightweight dynamic threshold container in order to prune the sparse network. By providing each filter with a suitable threshold for that filter at a low cost, a massive reduction in the number of parameters is achieved without affecting the contribution of certain pruning-sensitive layers to the network as a whole. In the final experiments, our two methods adapted to texture distribution were applied to ResNet Deep Neural Network (DNN) and VGG-16, which were deployed on the classical CIFAR-10/100 and ImageNet datasets with excellent results in order to facilitate comparison with good cutting-edge pruning methods. Code is available at https://github.com/wangyuzhe27/CDP-and-DTC.
