TY - JOUR
T1 - Sparse Gaussian Process Temporal Difference Learning for Marine Robot Navigation
AU - Martin, John
AU - Wang, Jinkun
AU - Englot, Brendan
N1 - Publisher Copyright:
© CoRL 2018. All rights reserved.
PY - 2018
Y1 - 2018
N2 - We present a method for Temporal Difference (TD) learning that addresses several challenges faced by robots learning to navigate in a marine environment. For improved data efficiency, our method reduces TD updates to Gaussian Process regression. To make predictions amenable to online settings, we introduce a sparse approximation with improved quality over current rejection-based methods. We derive the predictive value function posterior and use the moments to obtain a new algorithm for model-free policy evaluation, SPGP-SARSA. With simple changes, we show SPGP-SARSA can be reduced to a model-based equivalent, SPGP-TD. We perform comprehensive simulation studies and also conduct physical learning trials with an underwater robot. Our results show SPGP-SARSA can outperform the state-of-the-art sparse method, replicate the prediction quality of its exact counterpart, and be applied to solve underwater navigation tasks.
AB - We present a method for Temporal Difference (TD) learning that addresses several challenges faced by robots learning to navigate in a marine environment. For improved data efficiency, our method reduces TD updates to Gaussian Process regression. To make predictions amenable to online settings, we introduce a sparse approximation with improved quality over current rejection-based methods. We derive the predictive value function posterior and use the moments to obtain a new algorithm for model-free policy evaluation, SPGP-SARSA. With simple changes, we show SPGP-SARSA can be reduced to a model-based equivalent, SPGP-TD. We perform comprehensive simulation studies and also conduct physical learning trials with an underwater robot. Our results show SPGP-SARSA can outperform the state-of-the-art sparse method, replicate the prediction quality of its exact counterpart, and be applied to solve underwater navigation tasks.
KW - Reinforcement Learning
KW - Sparse Gaussian Process Regression
UR - http://www.scopus.com/inward/record.url?scp=85139584902&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85139584902&partnerID=8YFLogxK
M3 - Conference article
AN - SCOPUS:85139584902
VL - 87
SP - 179
EP - 189
JO - Proceedings of Machine Learning Research
JF - Proceedings of Machine Learning Research
T2 - 2nd Conference on Robot Learning, CoRL 2018
Y2 - 29 October 2018 through 31 October 2018
ER -