TY - GEN
T1 - RoMA
T2 - 12th IEEE International Conference on Cloud Networking, CloudNet 2023
AU - Tang, Xuting
AU - Xu, Jia
AU - Wang, Shusen
N1 - Publisher Copyright:
© 2023 IEEE.
PY - 2023
Y1 - 2023
N2 - This paper presents RoMA, a novel resilient Multi-Agent Reinforcement Learning (MARL) framework designed to handle dynamic participating agents during centralized training, addressing the limitations of standard MARL frameworks in accommodating agent variability and enabling efficient adaptation and training of agents, thus providing a scalable and flexible solution for model training and execution in cloud computing environments. For standard MARL frameworks, if new agents need to join or existing agents leave unexpectedly due to unreliable communication channels, standard MARL models need to be rebuilt and trained from scratch because of their structural limitations, which is very time-consuming. RoMA addresses this issue with a novel neural network architecture and a few-shot learning algorithm to enable the number of agents to vary during centralized training. When new agents join, RoMA can adapt all agents to the change in a few shots, and when agents leave the training process unexpectedly, RoMA can continue training the remaining agents without disruption.Our experiments demonstrate that RoMA is at least 70 times faster at adapting to new agents compared to baseline methods, and it can handle the leaving of agents without affecting the training of other agents. RoMA is applicable to a wide range of MARL settings, including cooperative, competitive, independent, and mixed environments.
AB - This paper presents RoMA, a novel resilient Multi-Agent Reinforcement Learning (MARL) framework designed to handle dynamic participating agents during centralized training, addressing the limitations of standard MARL frameworks in accommodating agent variability and enabling efficient adaptation and training of agents, thus providing a scalable and flexible solution for model training and execution in cloud computing environments. For standard MARL frameworks, if new agents need to join or existing agents leave unexpectedly due to unreliable communication channels, standard MARL models need to be rebuilt and trained from scratch because of their structural limitations, which is very time-consuming. RoMA addresses this issue with a novel neural network architecture and a few-shot learning algorithm to enable the number of agents to vary during centralized training. When new agents join, RoMA can adapt all agents to the change in a few shots, and when agents leave the training process unexpectedly, RoMA can continue training the remaining agents without disruption.Our experiments demonstrate that RoMA is at least 70 times faster at adapting to new agents compared to baseline methods, and it can handle the leaving of agents without affecting the training of other agents. RoMA is applicable to a wide range of MARL settings, including cooperative, competitive, independent, and mixed environments.
KW - Cloud Computing
KW - Multi-agent Reinforcement Learning
KW - Resilient Model
UR - http://www.scopus.com/inward/record.url?scp=85191266037&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85191266037&partnerID=8YFLogxK
U2 - 10.1109/CloudNet59005.2023.10490082
DO - 10.1109/CloudNet59005.2023.10490082
M3 - Conference contribution
AN - SCOPUS:85191266037
T3 - 2023 IEEE 12th International Conference on Cloud Networking, CloudNet 2023
SP - 247
EP - 255
BT - 2023 IEEE 12th International Conference on Cloud Networking, CloudNet 2023
Y2 - 1 November 2023 through 3 November 2023
ER -