Multi-Agent Hierarchical Reinforcement Learning for Remote IoT in the Space-Edge Cloud

Traditional terrestrial computing facilities, i.e., Edge Computing (EC) and Cloud Computing (CC), have limited applicability for remote Internet of Things (IoT) applications due to connectivity challenges. These challenges can be addressed through a Space-Edge Cloud continuum that leverages computing facilities enabled by Low Earth Orbit (LEO) satellites and terrestrial cloud facilities. We aim to minimize the joint latency and energy cost for processing remote IoT data in this continuum by formulating a complex optimization problem. Specifically, our goal is to optimize the complete offloading process by jointly selecting optimal routes, computation nodes, and buffering policies. We develop a Multi-Agent Hierarchical Reinforcement Learning (MA-HRL) solution to solve this problem. The proposed solution is developed in a Python environment and compared with several benchmark methods, demonstrating significant performance gains in terms of reduced latency and energy demands.