Efficient approximate dynamic programming based on design and analysis of computer experiments for infinite-horizon optimization

Ying Chen, Feng Liu, Jay M. Rosenberger, Victoria C.P. Chen, Asama Kulvanitchaiyanunt, Yuan Zhou

    Research output: Contribution to journalArticlepeer-review

    9 Scopus citations

    Abstract

    The approximate dynamic programming (ADP) method based on the design and analysis of computer experiments (DACE) approach has been demonstrated as an effective method to solve multistage decision-making problems in the literature. However, this method is still not efficient for infinite-horizon optimization considering the required large volume of sampling in the state space and high-quality value function identification. Therefore, we propose a sequential sampling algorithm and embed it into a DACE-based ADP method to obtain a high-quality value function approximation. Considering the limitations of the traditional stopping criterion (Bellman error bound), we further propose a 45-degree line stopping criterion to terminate value iteration early by identifying an optimally equivalent value function. A comparison of the computational results with those of other three existing policies indicates that the proposed sampling algorithm and stopping criterion can determine a high-quality ADP policy. Finally, we discuss the extrapolation issue of the value function approximated by multivariate adaptive regression splines, the results of which further demonstrate the quality of the ADP policy generated in this study.

    Original languageEnglish
    Article number105032
    JournalComputers and Operations Research
    Volume124
    DOIs
    StatePublished - Dec 2020

    Keywords

    • Approximate dynamic programming
    • Extrapolation
    • State space sampling
    • Stopping criterion

    Fingerprint

    Dive into the research topics of 'Efficient approximate dynamic programming based on design and analysis of computer experiments for infinite-horizon optimization'. Together they form a unique fingerprint.

    Cite this