A TWO-TIMESCALE REINFORCEMENT LEARNING APPROACH FOR CONTROL CO-DESIGN PROBLEMS

Eddieb Sadat, Mostaan Lotfalian Saremi, Alparslan Emrah Bayrak

    Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

    Abstract

    Design of smart (or active) systems that perform automated tasks intelligently based on the interaction with their environments requires a collective solution of the physical and control system design problems together. In this paper, we present a model-free on-policy reinforcement learning approach to solve control co-design problems for such smart systems. This approach uses a discrete two timescale reinforcement learning that addresses the control system design in an inner loop with a fast time scale and the physical system design in an outer loop with a slower time scale. Both design problems use the same temporal difference-based Q-learning formulation. We apply this two-time-scale reinforcement approach to the online video game EcoRacer where the physical system involves the design of a gear ratio for an electric vehicle and the control system involves acceleration and braking decisions over time to finish a track with minimum energy consumption within a limited time. The results show the ability of the proposed approach to find the system optimal solution for the EcoRacer case study within a reasonable computation time without requiring any knowledge of the physics governing the system. The proposed method is generalizable and has the potential to take advantage of the ongoing developments in the field of reinforcement learning.

    Original languageEnglish
    Title of host publication49th Design Automation Conference (DAC)
    ISBN (Electronic)9780791887301
    DOIs
    StatePublished - 2023
    EventASME 2023 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference, IDETC-CIE 2023 - Boston, United States
    Duration: 20 Aug 202323 Aug 2023

    Publication series

    NameProceedings of the ASME Design Engineering Technical Conference
    Volume3A

    Conference

    ConferenceASME 2023 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference, IDETC-CIE 2023
    Country/TerritoryUnited States
    CityBoston
    Period20/08/2323/08/23

    Keywords

    • Control co-design
    • model-free learning
    • reinforcement learning
    • video games

    Fingerprint

    Dive into the research topics of 'A TWO-TIMESCALE REINFORCEMENT LEARNING APPROACH FOR CONTROL CO-DESIGN PROBLEMS'. Together they form a unique fingerprint.

    Cite this