Multichannel cross-fusional convolutional neural networks

Article type: Research Article

Authors: Shan, Chuanhui^{a; *} | Ou, Jun^{b; 1} | Chen, Xiumei^{c; 1}

Affiliations: [a] College of Electrical Engineering, Anhui Polytechnic University, Wuhu, Anhui Province, China | [b] Hainan College of Software Technology, Qionghai, Hainan Province, China | [c] College of Biological and Food Engineering, Anhui Polytechnic University, Wuhu, Anhui Province, China

Correspondence: [*] Corresponding author. Chuanhui Shan, College of Electrical Engineering, Anhui Polytechnic University, Middle Beijing Road, Wuhu, 241000, Anhui Province, China. E-mail: chuanhshan@163.com.

Note: [1] These authors contributed equally to this work.

Abstract: As one of the main methods of information fusion, artificial intelligence class fusion algorithm not only inherits the powerful skills of artificial intelligence, but also inherits many advantages of information fusion. Similarly, as an important sub-field of artificial intelligence class fusion algorithm, deep learning class fusion algorithm also inherits advantages of deep learning and information fusion. Hence, deep learning fusion algorithm has become one of the research hotspots of many scholars. To solve the problem that the existing neural networks are input into multiple channels as a whole and cannot fully learn information of multichannel images, Shan et al. proposed multichannel concat-fusional convolutional neural networks. To mine more multichannel images’ information and further explore the performance of different fusion types, the paper proposes new fusional neural networks called multichannel cross-fusion convolutional neural networks (McCfCNNs) with fusion types of “R+G+B/R+G+B/R+G+B” and “R+G/G+B/B+R” based on the tremendous strengths of information fusion. Experiments show that McCfCNNs obtain 0.07-6.09% relative performance improvement in comparison with their corresponding non-fusion convolutional neural networks (CNNs) on diverse datasets (such as CIFAR100, SVHN, CALTECH256, and IMAGENET) under a certain computational complexity. Hence, McCfCNNs with fusion types of “R+G+B/R+G+B/R+G+B” and “R+G/G+B/B+R” can learn more fully multichannel images’ information, which provide a method and idea for processing multichannel information fusion, for example, remote sensing satellite images.

Keywords: Information fusion, fusion type “R+G+B/R+G+B/R+G+B”, fusion type “R+G/G+B/B+R”, CNN, McCfCNN

DOI: 10.3233/JIFS-224076

Journal: Journal of Intelligent & Fuzzy Systems, vol. 44, no. 6, pp. 10417-10436, 2023