TY - JOUR
T1 - A two-state partially observable Markov decision process with three actions
AU - Ben-Zvi, Tal
AU - Chernonog, Tatyana
AU - Avinadav, Tal
N1 - Publisher Copyright:
© 2016 Elsevier B.V. All rights reserved.
PY - 2016/11/1
Y1 - 2016/11/1
N2 - A process can be in either a stable or an unstable state interchangeably. The true state is unobservable and can only be inferred from observations. Three actions are available: continue with the process (CON), repair the process for a certain fee - bring the process to the stable state (REP), and obtain the state of the process for a cost (INS). The objective is to maximize the expected discounted value of the total future profits. We formulate the problem as a discrete-time Partially Observable Markov Decision Process (POMDP). We show that the expected profit function is convex and strictly increasing, and that the optimal policy has either one or two control limits. Also, we show that "dominance in expectation" (the expected revenue is larger in the stable state than in the unstable state) suffices for a control limit structure.
AB - A process can be in either a stable or an unstable state interchangeably. The true state is unobservable and can only be inferred from observations. Three actions are available: continue with the process (CON), repair the process for a certain fee - bring the process to the stable state (REP), and obtain the state of the process for a cost (INS). The objective is to maximize the expected discounted value of the total future profits. We formulate the problem as a discrete-time Partially Observable Markov Decision Process (POMDP). We show that the expected profit function is convex and strictly increasing, and that the optimal policy has either one or two control limits. Also, we show that "dominance in expectation" (the expected revenue is larger in the stable state than in the unstable state) suffices for a control limit structure.
KW - Control limits
KW - Decision processes
KW - Markov chains
KW - POMDP
UR - http://www.scopus.com/inward/record.url?scp=84971668209&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=84971668209&partnerID=8YFLogxK
U2 - 10.1016/j.ejor.2016.04.062
DO - 10.1016/j.ejor.2016.04.062
M3 - Article
AN - SCOPUS:84971668209
SN - 0377-2217
VL - 254
SP - 957
EP - 967
JO - European Journal of Operational Research
JF - European Journal of Operational Research
IS - 3
ER -