RoMA: Resilient Multi-Agent Reinforcement Learning with Dynamic Participating Agents

Xuting Tang, Jia Xu, Shusen Wang

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

This paper presents RoMA, a novel resilient Multi-Agent Reinforcement Learning (MARL) framework designed to handle dynamic participating agents during centralized training, addressing the limitations of standard MARL frameworks in accommodating agent variability and enabling efficient adaptation and training of agents, thus providing a scalable and flexible solution for model training and execution in cloud computing environments. For standard MARL frameworks, if new agents need to join or existing agents leave unexpectedly due to unreliable communication channels, standard MARL models need to be rebuilt and trained from scratch because of their structural limitations, which is very time-consuming. RoMA addresses this issue with a novel neural network architecture and a few-shot learning algorithm to enable the number of agents to vary during centralized training. When new agents join, RoMA can adapt all agents to the change in a few shots, and when agents leave the training process unexpectedly, RoMA can continue training the remaining agents without disruption.Our experiments demonstrate that RoMA is at least 70 times faster at adapting to new agents compared to baseline methods, and it can handle the leaving of agents without affecting the training of other agents. RoMA is applicable to a wide range of MARL settings, including cooperative, competitive, independent, and mixed environments.

Original languageEnglish
Title of host publication2023 IEEE 12th International Conference on Cloud Networking, CloudNet 2023
Pages247-255
Number of pages9
ISBN (Electronic)9798350313062
DOIs
StatePublished - 2023
Event12th IEEE International Conference on Cloud Networking, CloudNet 2023 - Hoboken, United States
Duration: 1 Nov 20233 Nov 2023

Publication series

Name2023 IEEE 12th International Conference on Cloud Networking, CloudNet 2023

Conference

Conference12th IEEE International Conference on Cloud Networking, CloudNet 2023
Country/TerritoryUnited States
CityHoboken
Period1/11/233/11/23

Keywords

  • Cloud Computing
  • Multi-agent Reinforcement Learning
  • Resilient Model

Fingerprint

Dive into the research topics of 'RoMA: Resilient Multi-Agent Reinforcement Learning with Dynamic Participating Agents'. Together they form a unique fingerprint.

Cite this