Affiliations: [a] College of Systems Engineering, National University of Defense Technology, Changsha, China
| [b] College of Computer, National University of Defense Technology, Changsha, China
Correspondence:
[*]
Corresponding authors. Xin Long and Xiangrong Zeng, College of Systems Engineering, National University of Defense Technology, Changsha 410073, China. (Xin Long, E-mail: longxin14@nudt.edu.cn); (Xiangrong Zeng, E-mail: zengxrong@gmail.com).
Abstract: The deployment of large-scale Convolutional Neural Networks (CNNs) in limited-power devices is hindered by their high computation cost and storage. In this paper, we propose a novel framework for CNNs to simultaneously achieve channel pruning and low-bit quantization by combining weight quantization with Sparse Group Lasso (SGL) regularization. We model this framework as a discretely constrained problem and solve it by Alternating Direction Method of Multipliers (ADMM). Different from previous approaches, the proposed method reduces not only model size but also computational operations. In experimental section, we evaluate the proposed framework on CIFAR datasets with several popular models such as VGG-7/16/19 and ResNet-18/34/50, which demonstrate that the proposed method can obtain low-bit networks and dramatically reduce redundant channels of the network with slight inference accuracy loss. Furthermore, we also visualize and analyze weight tensors, which showing the compact group-sparsity structure of them.
Keywords: Convolutional neural network (CNN), weight quantization, sparse group lasso (SGL), alternating direction method of multipliers (ADMM), channel pruning
