Market-Based Model in CR-IoT: A Q-Probabilistic Multi-Agent Reinforcement Learning Approach

The ever-increasing urban population and the corresponding material demands have brought unprecedented burdens to cities. To guarantee better QoS for citizens, smart cities leverage emerging technologies such as the Cognitive Radio Internet of Things (CR-IoT). However, resource allocation is a great challenge for CR-IoT, mainly because of the extremely numerous devices and users. Generally, the auction theory and game theory are applied to overcome the challenge. In this paper, we propose a multi-agent reinforcement learning (MARL) algorithm to learn the optimal resource allocation strategy in the oligopoly market model. Firstly, we model a multi-agent scenario with the primary users (PUs) as sellers and secondary users (SUs) as buyers. Then, we propose the Q-probabilistic multi-agent learning (QPML) and apply it to allocate resources in the market. In the multi-agent learning process, the PUs and SUs learn strategies to maximize their benefits and improve spectrum utilization. The performance of QPML is compared with Learning Automation (LA) through simulations. The experimental results show that our approach outperforms other approaches and performs well.