Affiliations: [a] School of Computer Science and Information Engineering, Hefei University of Technology, Hefei, China
| [b] Engineering Research Center of Safety Critical Industrial Measurement and Control Technology, Ministry of Education, Hefei University of Technology, Hefei, China
| [c] Anhui Mine IOT and Security Monitoring Technology Key Laboratory, Hefei, China
| [d] Intelligent Manufacturing Institute of Hefei University of Technology, Hefei, China
Abstract: End-to-end deep learning has gained considerable interests in autonomous driving vehicles. End-to-end autonomous driving uses the deep convolutional neural network to establish input-to-output mapping. However, existing end-to-end driving models only predict steering angle with front-facing camera data and poorly extract spatial-temporal information. Based on deep learning and attention mechanism, we propose an end-to-end driving model which combines the multi-stream attention module with the multi-stream network. As a multimodal multitask model, the proposed end-to-end driving model not only fully extracts spatial-temporal information from multimodality, but also adopts the multitask learning method with hard parameter sharing to predict the steering angle and speed. Furthermore, the proposed multi-stream attention module predicts the attention weights of streams based on the multimodal feature fusion, which encourages the proposed end-to-end driving model to pay attention to streams that positively impact the prediction result. We demonstrate the efficiency of the proposed driving model on the public Udacity dataset compared to existing models. Experimental results show that the proposed driving model has better performances than other existing methods.
