Optimal Cooperative Relaying and Power Control for IoUT Networks with Reinforcement Learning

Internet of Underwater Things (IoUT) consists of numerous sensor nodes distributed in an underwater area for sensing, collecting, processing information, and sending related messages to the data processing center. However, the characteristics of the underwater environment will bring strict limitations on communication coverage and power scarcity to IoUT networks. Applying cooperative communications to IoUT networks can expand the communication range and alleviate power shortages. In this article, we investigate the cooperative communication problem in a power-limited cooperative IoUT system and propose a reinforcement learning-based underwater relay selection strategy. Specifically, we first determine the optimal transmit powers of the source node and the selected underwater relay to maximize the end-to-end signal-to-noise ratio of the system. Then, we formulate the underwater cooperative relaying process as a Markov process and apply reinforcement learning to obtain an effective underwater relay selection strategy. The simulation results show that the performance of the proposed scheme outperforms that of the equal transmit power settings under the same conditions. In addition, the proposed deep Q-network-based underwater relay selection strategy improves the communication efficiency compared with the Q-learning-based strategy, and the number of iterations needed for convergence can be effectively reduced.