Collaborative Reinforcement Learning for Multi-Service Internet of Vehicles

Internet of Vehicles (IoV) is a recently introduced paradigm aiming at extending the Internet of Things (IoT) toward the vehicular scenario in order to cope with its specific requirements. Nowadays, there are several types of vehicles, with different characteristics, requested services, and delivered data types. In order to efficiently manage such heterogeneity, Edge Computing facilities are often deployed in the urban environment, usually co-located with the Roadside Units (RSUs), for creating what is referenced as Vehicular Edge Computing (VEC). In this paper, we consider a joint network selection and computation offloading optimization problem in multi-service VEC environments, aiming at minimizing the overall latency and the consumed energy in an IoV scenario. Two novel collaborative Q-learning based approaches are proposed, where Vehicle-to-Infrastructure (V2I) and Vehicle-to-Vehicle (V2V) communication paradigms are exploited, respectively. In the first approach, we define a collaborative Q-learning method in which, through V2I communications, several vehicles participate in the training process of a centralized Q-agent. In the second approach, by exploiting the V2V communications, each vehicle is made aware of the surrounding environment and the potential offloading neighbors, leading to better decisions in terms of network selection and offloading. In addition to the tabular method, an advanced deep learning-based approach is also used for the action value estimation, allowing to handle more complex vehicular scenarios. Simulation results show that the proposed approaches improve the network performance in terms of latency and consumed energy with respect to some benchmark solutions.