Affiliations: [a] School of Information Engineering, Jiangxi University of Science and Technology, Ganzhou, Jiangxi, China
| [b] School of Computing, Central South University, Changsha, Hunan, China
Correspondence:
[*]
Corresponding author. Yi-Min Mao, School of Information Engineering, Jiangxi University of Science and Technology, China. E-mail: mymlyc@163.com. and Zhi-Gang Chen, School of Computing, Central South University, China. E-mail: czg@csu.edu.cn.
Abstract: Deep convolutional neural networks (DCNNs), with their complex network structure and powerful feature learning and feature expression capabilities, have been remarkable successes in many large-scale recognition tasks. However, with the expectation of memory overhead and response time, along with the increasing scale of data, DCNN faces three non-rival challenges in a big data environment: excessive network parameters, slow convergence, and inefficient parallelism. To tackle these three problems, this paper develops a deep convolutional neural networks optimization algorithm (PDCNNO) in the MapReduce framework. The proposed method first pruned the network to obtain a compressed network in order to effectively reduce redundant parameters. Next, a conjugate gradient method based on modified secant equation (CGMSE) is developed in the Map phase to further accelerate the convergence of the network. Finally, a load balancing strategy based on regulate load rate (LBRLA) is proposed in the Reduce phase to quickly achieve equal grouping of data and thus improving the parallel performance of the system. We compared the PDCNNO algorithm with other algorithms on three datasets, including SVHN, EMNIST Digits, and ISLVRC2012. The experimental results show that our algorithm not only reduces the space and time overhead of network training but also obtains a well-performing speed-up ratio in a big data environment.
