TY - GEN
T1 - Designing optimal arbitrage policies for distributed energy systems in building clusters using reinforcement learning
AU - Odonkor, Philip
AU - Lewis, Kemper
N1 - Publisher Copyright:
Copyright © 2019 ASME.
PY - 2019
Y1 - 2019
N2 - In the wake of increasing proliferation of renewable energy and distributed energy resources (DERs), grid designers and operators alike are faced with several emerging challenges in curbing allocative grid inefficiencies and maintaining operational stability. One such challenge relates to the increased price volatility within real-time electricity markets, a result of the inherent intermittency of renewable energy. With this challenge, however, comes heightened economic interest in exploiting the arbitrage potential of price volatility towards demand-side energy cost savings. To this end, this paper aims to maximize the arbitrage value of electricity through the optimal design of control strategies for DERs. Formulated as an arbitrage maximization problem using design optimization, and solved using reinforcement learning, the proposed approach is applied towards shared DERs within multi-building residential clusters. We demonstrate its feasibility across three unique building cluster demand profiles, observing notable energy cost reductions over baseline values. This highlights a capability for generalized learning across multiple building clusters and the ability to design efficient arbitrage policies towards energy cost minimization. Finally, the approach is shown to be computationally tractable, designing efficient strategies in approximately 5 hours of training over a simulation time horizon of 1 month.
AB - In the wake of increasing proliferation of renewable energy and distributed energy resources (DERs), grid designers and operators alike are faced with several emerging challenges in curbing allocative grid inefficiencies and maintaining operational stability. One such challenge relates to the increased price volatility within real-time electricity markets, a result of the inherent intermittency of renewable energy. With this challenge, however, comes heightened economic interest in exploiting the arbitrage potential of price volatility towards demand-side energy cost savings. To this end, this paper aims to maximize the arbitrage value of electricity through the optimal design of control strategies for DERs. Formulated as an arbitrage maximization problem using design optimization, and solved using reinforcement learning, the proposed approach is applied towards shared DERs within multi-building residential clusters. We demonstrate its feasibility across three unique building cluster demand profiles, observing notable energy cost reductions over baseline values. This highlights a capability for generalized learning across multiple building clusters and the ability to design efficient arbitrage policies towards energy cost minimization. Finally, the approach is shown to be computationally tractable, designing efficient strategies in approximately 5 hours of training over a simulation time horizon of 1 month.
KW - Battery storage
KW - Building cluster
KW - Operational strategy design
KW - Reinforcement learning
UR - http://www.scopus.com/inward/record.url?scp=85076454451&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85076454451&partnerID=8YFLogxK
U2 - 10.1115/DETC2019-97190
DO - 10.1115/DETC2019-97190
M3 - Conference contribution
AN - SCOPUS:85076454451
T3 - Proceedings of the ASME Design Engineering Technical Conference
BT - 45th Design Automation Conference
T2 - ASME 2019 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference, IDETC-CIE 2019
Y2 - 18 August 2019 through 21 August 2019
ER -