Adaptive Assist-as-needed Control Based on Actor-Critic Reinforcement Learning

Yufeng Zhang, Shuai Li, Karen J. Nolan, Damiano Zanotto

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

24 Scopus citations

Abstract

In robot-assisted rehabilitation, assist-as-needed (AAN) controllers have been proposed to promote subjects' active participation, which is thought to lead to better training outcomes. Most of these AAN controllers require a patient-specific manual tuning of the parameters defining the underlying force-field, which typically results in a tedious and time-consuming process. In this paper, we propose a reinforcement-learning-based impedance controller that actively reshapes the stiffness of the force-field to the subject's performance, while providing assistance only when needed. This adaptability is made possible by correlating the subject's most recent performance to the ultimate control objective in real-time. In addition, the proposed controller is built upon action dependent heuristic dynamic programming using the actor-critic structure, and therefore does not require prior knowledge of the system model. The controller is experimentally validated with healthy subjects through a simulated ankle mobilization training session using a powered ankle-foot orthosis.

Original languageEnglish
Title of host publication2019 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2019
Pages4066-4071
Number of pages6
ISBN (Electronic)9781728140049
DOIs
StatePublished - Nov 2019
Event2019 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2019 - Macau, China
Duration: 3 Nov 20198 Nov 2019

Publication series

NameIEEE International Conference on Intelligent Robots and Systems
ISSN (Print)2153-0858
ISSN (Electronic)2153-0866

Conference

Conference2019 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2019
Country/TerritoryChina
CityMacau
Period3/11/198/11/19

Keywords

  • Assist-as-needed controller
  • rehabilitation robotics
  • reinforcement learning
  • robot-assisted training
  • wearable robotics

Fingerprint

Dive into the research topics of 'Adaptive Assist-as-needed Control Based on Actor-Critic Reinforcement Learning'. Together they form a unique fingerprint.

Cite this