TY - GEN
T1 - Hierarchical reinforcement learning framework for space exploration campaign design
AU - Chen, Hao
AU - Ho, Koki
N1 - Publisher Copyright:
© 2019, American Institute of Aeronautics and Astronautics Inc, AIAA. All rights reserved.
PY - 2019
Y1 - 2019
N2 - This paper proposes a reinforcement learning-based space campaign design framework to enable large-scale space logistics optimization for future human space exploration. The proposed method considers multi-mission space campaign as a Markov decision process, where decision makers perform mission planning based on newly observed events and anticipated future mission scenarios at the beginning of each space mission. In each mission, space mission planning decisions are considered in a multi-level hierarchical structure: 1) spacecraft design, 2) space infrastructure design, 3) detailed logistics actions. The first two levels mainly take into account mission interdependencies and are solved by the value function approximation and an actor-critic algorithm; whereas the lowest level finds the optimal space transportation planning to satisfy mission demands while conforming to the high-level decisions and are solved by the network-based space logistics optimization method. The proposed framework resolves the computational challenge in integrated space campaign design problems that consider space mission planning, space infrastructure design, and spacecraft design concurrently. A human lunar exploration space campaign is considered as the case study to evaluate the performance of the proposed method under stochastic space mission environments.
AB - This paper proposes a reinforcement learning-based space campaign design framework to enable large-scale space logistics optimization for future human space exploration. The proposed method considers multi-mission space campaign as a Markov decision process, where decision makers perform mission planning based on newly observed events and anticipated future mission scenarios at the beginning of each space mission. In each mission, space mission planning decisions are considered in a multi-level hierarchical structure: 1) spacecraft design, 2) space infrastructure design, 3) detailed logistics actions. The first two levels mainly take into account mission interdependencies and are solved by the value function approximation and an actor-critic algorithm; whereas the lowest level finds the optimal space transportation planning to satisfy mission demands while conforming to the high-level decisions and are solved by the network-based space logistics optimization method. The proposed framework resolves the computational challenge in integrated space campaign design problems that consider space mission planning, space infrastructure design, and spacecraft design concurrently. A human lunar exploration space campaign is considered as the case study to evaluate the performance of the proposed method under stochastic space mission environments.
UR - http://www.scopus.com/inward/record.url?scp=85095962614&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85095962614&partnerID=8YFLogxK
U2 - 10.2514/6.2019-4135
DO - 10.2514/6.2019-4135
M3 - Conference contribution
AN - SCOPUS:85095962614
SN - 9781624105906
T3 - AIAA Propulsion and Energy Forum and Exposition, 2019
BT - AIAA Propulsion and Energy Forum and Exposition, 2019
T2 - AIAA Propulsion and Energy Forum and Exposition, 2019
Y2 - 19 August 2019 through 22 August 2019
ER -