Machine learning-based reduction and unbiased reinforcement learning method for QoS-aware services composition in IoT environments
摘要
The selection and composition of competitive functional services to satisfy the user’s evolving needs is an open challenge due to the increasing number of services offered by applications in internet of things (IoT) environments. This challenge can be handled using optimization methods based on machine learning (ML) and software agent models to find the most appropriate services in terms of quality of service (QoS) criteria values. Among these methods, several reinforcement learning (RL) techniques were exploited in the literature. However, most of them cause biased learning and consider the overall set of services during the composition process. Indeed, biased learning may lead the RL-based approaches to a convergence failure of the composition mechanism. This failure along with considering the overall set of services induce a performance degradation in terms of composition time and cumulative reward especially when the services space becomes larger. This paper proposes to combine machine learning-based reduction techniques with the Monte-Carlo method for services composition (ML-MCSC) in IoT environments. Unlike most existing services composition algorithms that consider the overall set of services, the ML-MCSC approach exploits two ML techniques (random forest and t-distributed stochastic neighbor embedding) to reduce the number of services that will be considered in the composition process. This reduction improves the QoS cumulative reward and decreases the composition time since the unpromising services in terms of QoS are filtered out. Furthermore, the services composition is modeled as a Markov decision process (MDP) and solved using the unbiased on-policy Monte Carlo method with adaptive decayed