Method for intelligent obstacle avoidance decision-making of unmanned vessel in unknown waters

WANG Chengbo1，ZHANG Xinyu1，ZHANG Jiawei1，LIU Shuo2 1 Key Laboratory of Marine Simulation and Control for Ministry of Communications，Dalian Maritime University， Dalian 116026，China 2 College of Navigation，Dalian Maritime University，Dalian 116026，China

Abstract：［Objectives］In order to realize intelligent obstacle avoidance of unmanned vessel in unknown waters，［Methods］ an intelligent obstacle avoidance decision-making model of the unmanned vessel based on Deep Reinforcement Learning（DRL）is established. Here we analyze the problems encountered in the unmanned vessel's intelligent obstacle avoidance decision-making，propose the design criteria of the intelligent obstacle avoidance decision-making，and then accordingly establish a decision-making model based on Markov Decision Process （MDP），through which obtain the optimal strategy by value function to make the maximum returns in behavior mapping of the unmanned vessel status and to design an excitation function specially composed of target approaching，off course and safety. Finally，carry out the simulation tests respectively in static and dynamic waters.［Results］The results show that the proposed intelligent decision-making method can effectively avoid obstacles，and ensure the safe navigation of the unmanned vessel in unknown waters. ［Conclusions］ The proposed method can provide a theoretical reference for autonomous navigation of the unmanned vessel. Key words：unmanned vessel；intelligent decision-making；Deep Reinforcement Learning（DRL）； obstacle avoidance