Reinforcement Learning-based Adaptive Trajectory Planning for AUVs in Under-ice Environments

Chaofeng Wang, Li Wei, Zhaohui Wang, Min Song, Nina Mahmoudian

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

3 Scopus citations

Abstract

This work studies online learning-based trajectory planning for multiple autonomous underwater vehicles (AUVs) to estimate a water parameter field of interest in the under-ice environment. A centralized system is considered, where several fixed access points (APs) on the ice layer are introduced as gateways for communications between the AUVs and a remote data fusion center (FC). We model the water parameter field of interest as a Gaussian process (GP) with unknown hyper-parameters. The AUV trajectories for sampling are determined on an epoch-by-epoch basis. At the end of each epoch, the APs relay the observed field samples from all the AUVs to the FC which computes the posterior distribution of the field based on the Gaussian process regression (GPR) and estimates the field hyper-parameters. The optimal trajectories of all the AUVs in the next epoch are determined to minimize a long-term cost that is defined based on the field uncertainty reduction and the AUV mobility cost, subject to the kinematics constraint, the communication range constraint and the sensing area constraint. We formulate the adaptive trajectory planning problem as a Markov decision process (MDP). A reinforcement learning (RL)-based online learning method is designed to determine the optimal AUV trajectories in a constrained continuous space. Simulation results show that the proposed learning-based trajectory planning algorithm has performance similar to a benchmark method that assumes perfect knowledge of the field hyper-parameters.

Original languageEnglish
Title of host publicationOCEANS 2018 MTS/IEEE Charleston, OCEAN 2018
ISBN (Electronic)9781538648148
DOIs
StatePublished - 7 Jan 2019
EventOCEANS 2018 MTS/IEEE Charleston, OCEANS 2018 - Charleston, United States
Duration: 22 Oct 201825 Oct 2018

Publication series

NameOCEANS 2018 MTS/IEEE Charleston, OCEAN 2018

Conference

ConferenceOCEANS 2018 MTS/IEEE Charleston, OCEANS 2018
Country/TerritoryUnited States
CityCharleston
Period22/10/1825/10/18

Fingerprint

Dive into the research topics of 'Reinforcement Learning-based Adaptive Trajectory Planning for AUVs in Under-ice Environments'. Together they form a unique fingerprint.

Cite this