TY - JOUR
T1 - Hierarchical Reinforcement Learning Framework for Stochastic Spaceflight Campaign Design
AU - Takubo, Yuji
AU - Chen, Hao
AU - Ho, Koki
N1 - Publisher Copyright:
© AIAA International. All rights reserved.
PY - 2022
Y1 - 2022
N2 - This paper develops a hierarchical reinforcement learning architecture for multimission spaceflight campaign design under uncertainty, including vehicle design, infrastructure deployment planning, and space transportation scheduling. This problem involves a high-dimensional design space and is challenging especially with uncertainty present. To tackle this challenge, the developed framework has a hierarchical structure with reinforcement learning and network-based mixed-integer linear programming (MILP), where the former optimizes campaign-level decisions (e.g., design of the vehicle used throughout the campaign, destination demand assigned to each mission in the campaign), whereas the latter optimizes the detailed mission-level decisions (e.g., when to launch what from where to where). The framework is applied to a set of human lunar exploration campaign scenarios with uncertain in situ resource utilization performance as a case study. The main value of this work is its integration of the rapidly growing reinforcement learning research and the existing MILP-based space logistics methods through a hierarchical framework to handle the otherwise intractable complexity of space mission design under uncertainty. This unique framework is expected to be a critical steppingstone for the emerging research direction of artificial intelligence for space mission design.
AB - This paper develops a hierarchical reinforcement learning architecture for multimission spaceflight campaign design under uncertainty, including vehicle design, infrastructure deployment planning, and space transportation scheduling. This problem involves a high-dimensional design space and is challenging especially with uncertainty present. To tackle this challenge, the developed framework has a hierarchical structure with reinforcement learning and network-based mixed-integer linear programming (MILP), where the former optimizes campaign-level decisions (e.g., design of the vehicle used throughout the campaign, destination demand assigned to each mission in the campaign), whereas the latter optimizes the detailed mission-level decisions (e.g., when to launch what from where to where). The framework is applied to a set of human lunar exploration campaign scenarios with uncertain in situ resource utilization performance as a case study. The main value of this work is its integration of the rapidly growing reinforcement learning research and the existing MILP-based space logistics methods through a hierarchical framework to handle the otherwise intractable complexity of space mission design under uncertainty. This unique framework is expected to be a critical steppingstone for the emerging research direction of artificial intelligence for space mission design.
UR - http://www.scopus.com/inward/record.url?scp=85127484695&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85127484695&partnerID=8YFLogxK
U2 - 10.2514/1.A35122
DO - 10.2514/1.A35122
M3 - Article
AN - SCOPUS:85127484695
SN - 0022-4650
VL - 59
SP - 421
EP - 433
JO - Journal of Spacecraft and Rockets
JF - Journal of Spacecraft and Rockets
IS - 2
ER -