TY - JOUR
T1 - Market-Based Model in CR-IoT
T2 - A Q-Probabilistic Multi-Agent Reinforcement Learning Approach
AU - Wang, Dan
AU - Zhang, Wei
AU - Song, Bin
AU - Du, Xiaojiang
AU - Guizani, Mohsen
N1 - Publisher Copyright:
© 2015 IEEE.
PY - 2020/3
Y1 - 2020/3
N2 - The ever-increasing urban population and the corresponding material demands have brought unprecedented burdens to cities. To guarantee better QoS for citizens, smart cities leverage emerging technologies such as the Cognitive Radio Internet of Things (CR-IoT). However, resource allocation is a great challenge for CR-IoT, mainly because of the extremely numerous devices and users. Generally, the auction theory and game theory are applied to overcome the challenge. In this paper, we propose a multi-agent reinforcement learning (MARL) algorithm to learn the optimal resource allocation strategy in the oligopoly market model. Firstly, we model a multi-agent scenario with the primary users (PUs) as sellers and secondary users (SUs) as buyers. Then, we propose the Q-probabilistic multi-agent learning (QPML) and apply it to allocate resources in the market. In the multi-agent learning process, the PUs and SUs learn strategies to maximize their benefits and improve spectrum utilization. The performance of QPML is compared with Learning Automation (LA) through simulations. The experimental results show that our approach outperforms other approaches and performs well.
AB - The ever-increasing urban population and the corresponding material demands have brought unprecedented burdens to cities. To guarantee better QoS for citizens, smart cities leverage emerging technologies such as the Cognitive Radio Internet of Things (CR-IoT). However, resource allocation is a great challenge for CR-IoT, mainly because of the extremely numerous devices and users. Generally, the auction theory and game theory are applied to overcome the challenge. In this paper, we propose a multi-agent reinforcement learning (MARL) algorithm to learn the optimal resource allocation strategy in the oligopoly market model. Firstly, we model a multi-agent scenario with the primary users (PUs) as sellers and secondary users (SUs) as buyers. Then, we propose the Q-probabilistic multi-agent learning (QPML) and apply it to allocate resources in the market. In the multi-agent learning process, the PUs and SUs learn strategies to maximize their benefits and improve spectrum utilization. The performance of QPML is compared with Learning Automation (LA) through simulations. The experimental results show that our approach outperforms other approaches and performs well.
KW - CR-IoT
KW - MARL
KW - market model
KW - resource allocation
UR - http://www.scopus.com/inward/record.url?scp=85074500247&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85074500247&partnerID=8YFLogxK
U2 - 10.1109/TCCN.2019.2950242
DO - 10.1109/TCCN.2019.2950242
M3 - Article
AN - SCOPUS:85074500247
VL - 6
SP - 179
EP - 188
JO - IEEE Transactions on Cognitive Communications and Networking
JF - IEEE Transactions on Cognitive Communications and Networking
IS - 1
M1 - 8887219
ER -