Hierarchical reinforcement learning framework for space exploration campaign design

Hao Chen, Koki Ho

    Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

    1 Scopus citations

    Abstract

    This paper proposes a reinforcement learning-based space campaign design framework to enable large-scale space logistics optimization for future human space exploration. The proposed method considers multi-mission space campaign as a Markov decision process, where decision makers perform mission planning based on newly observed events and anticipated future mission scenarios at the beginning of each space mission. In each mission, space mission planning decisions are considered in a multi-level hierarchical structure: 1) spacecraft design, 2) space infrastructure design, 3) detailed logistics actions. The first two levels mainly take into account mission interdependencies and are solved by the value function approximation and an actor-critic algorithm; whereas the lowest level finds the optimal space transportation planning to satisfy mission demands while conforming to the high-level decisions and are solved by the network-based space logistics optimization method. The proposed framework resolves the computational challenge in integrated space campaign design problems that consider space mission planning, space infrastructure design, and spacecraft design concurrently. A human lunar exploration space campaign is considered as the case study to evaluate the performance of the proposed method under stochastic space mission environments.

    Original languageEnglish
    Title of host publicationAIAA Propulsion and Energy Forum and Exposition, 2019
    DOIs
    StatePublished - 2019
    EventAIAA Propulsion and Energy Forum and Exposition, 2019 - Indianapolis, United States
    Duration: 19 Aug 201922 Aug 2019

    Publication series

    NameAIAA Propulsion and Energy Forum and Exposition, 2019

    Conference

    ConferenceAIAA Propulsion and Energy Forum and Exposition, 2019
    Country/TerritoryUnited States
    CityIndianapolis
    Period19/08/1922/08/19

    Fingerprint

    Dive into the research topics of 'Hierarchical reinforcement learning framework for space exploration campaign design'. Together they form a unique fingerprint.

    Cite this