TY - GEN
T1 - Learning without Gradients
T2 - Artificial Intelligence and Machine Learning in Defense Applications IV 2022
AU - Morcos, Amir
AU - Man, Hong
AU - West, Aaron
AU - Maguire, Brian
N1 - Publisher Copyright:
© 2022 SPIE.
PY - 2022
Y1 - 2022
N2 - The field of Reinforcement Learning continues to show promise in solving old problems in new innovative ways. Thanks to the algorithms' ability to learn without an explicit set of labeled training data, the action, environment, reward approach has lured many researches into framing old problems in this manner. Recent publications have demonstrated how utilizing a multi-agent reinforcement learning approach can lead to a superior policy for optimization algorithm over the current standards. The challenge with the aforementioned approaches is the inclusion of the gradient in the state-space. This forces a costly calculation that is often the bottle neck in most machine learning problems, often limiting or preventing training at the edge or on the front lines. While previous works dating back decades have demonstrated the ability to train simple machine learning models without the use of gradients, none have done so using a policy which leverages previous experiences to solve the problem more quickly. This work will show how a Multi-Agent Reinforcement Learning approach can be used to optimize models in training without the need for the gradient of the loss function, effectively eliminating the need for backpropagation and significantly reducing the computational power required to train a model. Furthermore, the work will examine conditions under which the agents failed to find an optimal solution. As well as how this approach can be beneficial in complex defense applications.
AB - The field of Reinforcement Learning continues to show promise in solving old problems in new innovative ways. Thanks to the algorithms' ability to learn without an explicit set of labeled training data, the action, environment, reward approach has lured many researches into framing old problems in this manner. Recent publications have demonstrated how utilizing a multi-agent reinforcement learning approach can lead to a superior policy for optimization algorithm over the current standards. The challenge with the aforementioned approaches is the inclusion of the gradient in the state-space. This forces a costly calculation that is often the bottle neck in most machine learning problems, often limiting or preventing training at the edge or on the front lines. While previous works dating back decades have demonstrated the ability to train simple machine learning models without the use of gradients, none have done so using a policy which leverages previous experiences to solve the problem more quickly. This work will show how a Multi-Agent Reinforcement Learning approach can be used to optimize models in training without the need for the gradient of the loss function, effectively eliminating the need for backpropagation and significantly reducing the computational power required to train a model. Furthermore, the work will examine conditions under which the agents failed to find an optimal solution. As well as how this approach can be beneficial in complex defense applications.
UR - http://www.scopus.com/inward/record.url?scp=85145233321&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85145233321&partnerID=8YFLogxK
U2 - 10.1117/12.2636231
DO - 10.1117/12.2636231
M3 - Conference contribution
AN - SCOPUS:85145233321
T3 - Proceedings of SPIE - The International Society for Optical Engineering
BT - Artificial Intelligence and Machine Learning in Defense Applications IV
A2 - Dijk, Judith
Y2 - 6 September 2022 through 7 September 2022
ER -