Affiliations: [a] School of Computer Science and Technology, Wuhan University of Technology, Wuhan, PR China | [b] Intelligent Transportation System Centre, Wuhan University of Technology, Wuhan, PR China | [c]
National Engineering Research Centre for Water Transport Safety, Wuhan, PR China | [d] Department of Maritime and Transport Technology, Delft University of Technology, Delft, the Netherlands | [e] Department of Mechanical Engineering, University College London, Torrington Place, London, UK
Correspondence:
[*]
Corresponding author. Feng Ma, Intelligent Transportation System Centre, Wuhan University of Technology, Wuhan,
430063, China. E-mail: martin7wind@whut.edu.cn.
Abstract: Human experience is regarded as an indispensable part of artificial intelligence in the process of controlling or decision making for autonomous cargo ships. In this paper, a novel Deep Q-Network-based (DQN) approach is proposed, which performs satisfactorily in controlling a cargo ship automatically without any human experience. At the very beginning, we use the model of KRISO Very Large Crude Carrier (KVLCC2) to describe a cargo ship. To manipulate this ship has to conquer great inertia and relatively insufficient driving force. Subsequently, customary waterways, regulations, conventions are described with Artificial Potential Field and value-functions in DQN. Based on this, the artificial intelligence of planning and controlling a cargo ship can be obtained by undertaking sufficient training, which can control the ship directly, while avoiding collisions, keeping its position in the middle of the route as much as possible. In simulation experiments, it is demonstrated that such an approach performs better than manual works and other traditional methods in most conditions, which makes the proposed method a promising solution in improving the autonomy level of cargo ships.
Keywords: Deep Q-network, reinforcement learning, artificial intelligence, autonomous ships
