TY - GEN
T1 - Interactive learning from policy-dependent human feedback
AU - MacGlashan, James
AU - Ho, Mark K.
AU - Loftin, Robert
AU - Peng, Bel
AU - Wang, Guan
AU - Roberts, David L.
AU - Taylor, Matthew E.
AU - Littman, Michael L.
N1 - Publisher Copyright:
© 2017 by the author(s).
PY - 2017
Y1 - 2017
N2 - This paper investigates the problem of interactively learning behaviors communicated by a human teacher using positive and negative feedback. Much previous work on this problem has made the assumption that people provide feedback for decisions that is dependent on the behavior they are teaching and is independent from the learner's current policy. We present empirical results that show this assumption to be false- whether human trainers give a positive or negative feedback for a decision is influenced by the learner's current policy. Based on this insight, we introduce Convergent Actor-Critic by Humans (COACH), an algorithm for learning from policy-dependent feedback that converges to a local optimum. Finally, we demonstrate that COACH can successfully learn multiple behaviors on a physical robot.
AB - This paper investigates the problem of interactively learning behaviors communicated by a human teacher using positive and negative feedback. Much previous work on this problem has made the assumption that people provide feedback for decisions that is dependent on the behavior they are teaching and is independent from the learner's current policy. We present empirical results that show this assumption to be false- whether human trainers give a positive or negative feedback for a decision is influenced by the learner's current policy. Based on this insight, we introduce Convergent Actor-Critic by Humans (COACH), an algorithm for learning from policy-dependent feedback that converges to a local optimum. Finally, we demonstrate that COACH can successfully learn multiple behaviors on a physical robot.
UR - http://www.scopus.com/inward/record.url?scp=85040863039&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85040863039&partnerID=8YFLogxK
M3 - Conference contribution
AN - SCOPUS:85040863039
T3 - 34th International Conference on Machine Learning, ICML 2017
SP - 3557
EP - 3566
BT - 34th International Conference on Machine Learning, ICML 2017
T2 - 34th International Conference on Machine Learning, ICML 2017
Y2 - 6 August 2017 through 11 August 2017
ER -