Zero-Shot Reinforcement Learning on Graphs for Autonomous Exploration under Uncertainty

Fanfei Chen, Paul Szenher, Yewei Huang, Jinkun Wang, Tixiao Shan, Shi Bai, Brendan Englot

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

12 Scopus citations

Abstract

This paper studies the problem of autonomous exploration under localization uncertainty for a mobile robot with 3D range sensing. We present a framework for self-learning a high-performance exploration policy in a single simulation environment, and transferring it to other environments, which may be physical or virtual. Recent work in transfer learning achieves encouraging performance by domain adaptation and domain randomization to expose an agent to scenarios that fill the inherent gaps in sim2sim and sim2real approaches. However, it is inefficient to train an agent in environments with randomized conditions to learn the important features of its current state. An agent can use domain knowledge provided by human experts to learn efficiently. We propose a novel approach that uses graph neural networks in conjunction with deep reinforcement learning, enabling decision-making over graphs containing relevant exploration information provided by human experts to predict a robot's optimal sensing action in belief space. The policy, which is trained only in a single simulation environment, offers a real-time, scalable, and transferable decision-making strategy, resulting in zero-shot transfer to other simulation environments and even real-world environments.

Original languageEnglish
Title of host publication2021 IEEE International Conference on Robotics and Automation, ICRA 2021
Pages5193-5199
Number of pages7
ISBN (Electronic)9781728190778
DOIs
StatePublished - 2021
Event2021 IEEE International Conference on Robotics and Automation, ICRA 2021 - Xi'an, China
Duration: 30 May 20215 Jun 2021

Publication series

NameProceedings - IEEE International Conference on Robotics and Automation
Volume2021-May
ISSN (Print)1050-4729

Conference

Conference2021 IEEE International Conference on Robotics and Automation, ICRA 2021
Country/TerritoryChina
CityXi'an
Period30/05/215/06/21

Fingerprint

Dive into the research topics of 'Zero-Shot Reinforcement Learning on Graphs for Autonomous Exploration under Uncertainty'. Together they form a unique fingerprint.

Cite this